docs: add vulnerability database contribution guide (#9667)

This commit is contained in:
Teppei Fukuda
2025-10-16 09:11:03 +04:00
committed by GitHub
parent 8e6a7ff670
commit fcd8dcdd1d
3 changed files with 233 additions and 0 deletions

View File

@@ -0,0 +1,144 @@
# Add Vulnerability Advisory Source
This guide walks through the process of adding a new vulnerability advisory source to Trivy.
!!! info
For an overview of how Trivy's vulnerability database works, see the [Overview](overview.md) page.
## Prerequisites
Before starting, ensure you have:
1. Identified the upstream advisory source and its API/format
2. Checked that the data source doesn't already exist in Trivy
3. Created a GitHub discussion or issue to discuss the addition
## Required Changes
To add a new vulnerability advisory source, you'll need to make changes across three repositories. Below we'll use the Echo OS support as an example.
### Step 1: Add Fetcher Script (vuln-list-update)
!!! note
Skip this step if your advisory source is already managed in a Git repository (e.g., GitHub, GitLab).
Create a fetcher script in [vuln-list-update] to collect advisories from the upstream source.
**Key tasks:**
- Fetch advisories from the upstream API or source
- Validate the advisory format and data
- Save advisories as JSON files in the [vuln-list] directory structure
- **Store original data as-is where possible**: Avoid preprocessing or modifying advisory fields. Save the raw data exactly as provided by the upstream source (format conversion like YAML to JSON is acceptable for consistency)
- Include all necessary metadata (CVE ID, affected versions, severity, etc.)
**Example PR:**
- [feat(echo): Add Echo Support (vuln-list-update#350)](https://github.com/aquasecurity/vuln-list-update/pull/350)
### Step 2: Add Parser (trivy-db)
Create a parser in [trivy-db] to transform raw advisories into Trivy's database format.
**Key tasks:**
- Create a new vulnerability source in `pkg/vulnsrc/`
- Implement the advisory parsing logic
- Map advisory fields to Trivy's vulnerability schema
- Handle version ranges and affected packages correctly
- Store CVE mappings if available
- Add unit tests for the parser
**Example PR:**
- [feat(echo): Add Echo Support (trivy-db#528)](https://github.com/aquasecurity/trivy-db/pull/528)
### Step 3: Add OS/Ecosystem Support (Trivy)
Update [trivy] to support the new operating system or package ecosystem.
**Key tasks:**
- Add OS analyzer in `pkg/fanal/analyzer/os/` to detect the OS
- Implement vulnerability detection logic if special handling is needed
- Add integration tests with test data
- Update documentation to include the new data source
**Example PR:**
- [feat(echo): Add Echo Support (trivy#8833)](https://github.com/aquasecurity/trivy/pull/8833)
## Complete Example: Echo OS Support
The Echo OS support was added through three coordinated PRs:
1. **vuln-list-update**: Fetches Echo advisories from `https://advisory.echohq.com/data.json`
- PR: https://github.com/aquasecurity/vuln-list-update/pull/350
2. **trivy-db**: Parses Echo advisories and stores them in the database
- PR: https://github.com/aquasecurity/trivy-db/pull/528
3. **Trivy**: Detects Echo OS and scans for vulnerabilities
- PR: https://github.com/aquasecurity/trivy/pull/8833
## Testing Your Changes
### Test vuln-list-update
First, fetch all existing advisories (required for building the database):
```bash
cd vuln-list-update
go run main.go -vuln-list-dir /path/to/vuln-list
```
Then, test your new data source by fetching only your target:
```bash
go run main.go -target your-source -vuln-list-dir /path/to/vuln-list
```
Verify that advisories are correctly saved in the vuln-list directory.
### Test trivy-db
```bash
cd trivy-db
make db-build CACHE_DIR=/path/to/cache
```
Check that the database is built without errors and contains your advisories.
!!! note
The `CACHE_DIR` should point to the parent directory of your vuln-list directory. For example, if your vuln-list is at `/tmp/test/vuln-list`, set `CACHE_DIR=/tmp/test`.
You can inspect the built database using BoltDB viewer tools like [boltwiz](https://github.com/Moniseeta/boltwiz):
```bash
# Open the database
boltwiz out/trivy.db
```
This allows you to verify that your vulnerabilities are correctly stored in the database.
### Test Trivy
```bash
# Build Trivy with your changes
mage build
# Use your local database
./trivy image --skip-db-update --cache-dir /path/to/cache your-test-image
```
Verify that vulnerabilities from your new data source are detected correctly.
## Getting Help
If you have questions or need help:
1. Check existing data sources for reference implementations
2. [Start a discussion](https://github.com/aquasecurity/trivy/discussions/new) in the Trivy repository
[vuln-list]: https://github.com/aquasecurity/vuln-list
[vuln-list-update]: https://github.com/aquasecurity/vuln-list-update
[trivy-db]: https://github.com/aquasecurity/trivy-db
[trivy]: https://github.com/aquasecurity/trivy

View File

@@ -0,0 +1,86 @@
# Vulnerability Data Sources
This section explains how Trivy's vulnerability database works and how to contribute new advisory data sources.
## Overview
Trivy's vulnerability database is built through a multi-repository workflow involving three main repositories:
```mermaid
graph LR
A[Advisory Sources] -->|vuln-list-update| B[vuln-list]
B --> C["trivy-db<br/>(Trivy DB)"]
C --> D["trivy<br/>(Trivy CLI)"]
E[GitHub-managed<br/>Advisories] --> C
```
### Workflow Steps
1. **Advisory Collection** ([vuln-list-update])
- Fetch raw advisories from upstream sources
- Store them in [vuln-list] repository
- Run periodically via cron to keep advisories up-to-date
- This step can be skipped if advisories are already managed in a Git repository (e.g., GitHub Security Advisories)
2. **Database Build** ([trivy-db])
- Parse advisories from [vuln-list] or directly from Git-managed sources
- Transform them into Trivy's database format
- Publish the built database periodically via cron
3. **Database Consumption** ([trivy])
- Download the latest vulnerability database at scan time
- Use it to detect vulnerabilities in scan targets
## Why Store Advisories in vuln-list?
For data sources that are not already Git-managed, storing advisories in the [vuln-list] repository provides several benefits:
- **Transparency**: Easy to track changes and differences between advisory versions
- **Web UI**: Browse advisories directly on GitHub with a user-friendly interface
- **Stability**: Mitigate issues when upstream advisory servers are unstable or unavailable
- **Shareability**: Provide stable URLs to reference specific advisories
- **Data Quality**: Validate advisory data before committing to vuln-list, preventing malformed data or unexpected format changes from breaking Trivy DB
- **Historical Data**: Preserve past advisories when upstream formats change
## Repository Overview
### [vuln-list-update]
This repository contains scripts that fetch advisories from various upstream sources. Each data source has its own package that handles:
- Fetching advisories from APIs or web sources
- Validating the advisory format and data
- Saving them to the [vuln-list] repository
### [vuln-list]
This repository serves as a data storage for raw advisories fetched by [vuln-list-update]. Key characteristics:
- Contains raw advisory data in JSON format
- Updated automatically by [vuln-list-update] scripts via cron
- **Not for manual contributions**: Direct pull requests to this repository are not accepted
- Used as the source for [trivy-db] to build the vulnerability database
### [trivy-db]
This repository contains parsers that transform raw advisories into Trivy's database format. Each data source has its own vulnerability source handler that:
- Reads advisory files from [vuln-list] or directly from Git-managed sources (e.g., GitHub Security Advisories)
- Maps advisory fields to Trivy's schema
- Stores vulnerability information in the database
### [trivy]
The main Trivy repository contains:
- OS and package analyzers to detect what's installed
- Vulnerability detection logic
## Next Steps
Ready to add a new vulnerability advisory source? See the [Add Vulnerability Advisory Source](add-vulnerability-source.md) guide for detailed steps.
[vuln-list]: https://github.com/aquasecurity/vuln-list
[vuln-list-update]: https://github.com/aquasecurity/vuln-list-update
[trivy-db]: https://github.com/aquasecurity/trivy-db
[trivy]: https://github.com/aquasecurity/trivy

View File

@@ -232,6 +232,9 @@ nav:
- Contribute Rego Checks:
- Overview: community/contribute/checks/overview.md
- Add Service Support: community/contribute/checks/service-support.md
- Contribute Vulnerability Data Sources:
- Overview: community/contribute/vulnerability-database/overview.md
- Add Vulnerability Advisory Source: community/contribute/vulnerability-database/add-vulnerability-source.md
- Maintainer:
- PR Review: community/maintainer/pr-review.md
- Release Flow: community/maintainer/release-flow.md