mirror of
https://github.com/aquasecurity/trivy.git
synced 2025-12-12 15:50:15 -08:00
docs: add vulnerability database contribution guide (#9667)
This commit is contained in:
@@ -0,0 +1,144 @@
|
||||
# Add Vulnerability Advisory Source
|
||||
|
||||
This guide walks through the process of adding a new vulnerability advisory source to Trivy.
|
||||
|
||||
!!! info
|
||||
For an overview of how Trivy's vulnerability database works, see the [Overview](overview.md) page.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before starting, ensure you have:
|
||||
|
||||
1. Identified the upstream advisory source and its API/format
|
||||
2. Checked that the data source doesn't already exist in Trivy
|
||||
3. Created a GitHub discussion or issue to discuss the addition
|
||||
|
||||
## Required Changes
|
||||
|
||||
To add a new vulnerability advisory source, you'll need to make changes across three repositories. Below we'll use the Echo OS support as an example.
|
||||
|
||||
### Step 1: Add Fetcher Script (vuln-list-update)
|
||||
|
||||
!!! note
|
||||
Skip this step if your advisory source is already managed in a Git repository (e.g., GitHub, GitLab).
|
||||
|
||||
Create a fetcher script in [vuln-list-update] to collect advisories from the upstream source.
|
||||
|
||||
**Key tasks:**
|
||||
|
||||
- Fetch advisories from the upstream API or source
|
||||
- Validate the advisory format and data
|
||||
- Save advisories as JSON files in the [vuln-list] directory structure
|
||||
- **Store original data as-is where possible**: Avoid preprocessing or modifying advisory fields. Save the raw data exactly as provided by the upstream source (format conversion like YAML to JSON is acceptable for consistency)
|
||||
- Include all necessary metadata (CVE ID, affected versions, severity, etc.)
|
||||
|
||||
**Example PR:**
|
||||
|
||||
- [feat(echo): Add Echo Support (vuln-list-update#350)](https://github.com/aquasecurity/vuln-list-update/pull/350)
|
||||
|
||||
### Step 2: Add Parser (trivy-db)
|
||||
|
||||
Create a parser in [trivy-db] to transform raw advisories into Trivy's database format.
|
||||
|
||||
**Key tasks:**
|
||||
|
||||
- Create a new vulnerability source in `pkg/vulnsrc/`
|
||||
- Implement the advisory parsing logic
|
||||
- Map advisory fields to Trivy's vulnerability schema
|
||||
- Handle version ranges and affected packages correctly
|
||||
- Store CVE mappings if available
|
||||
- Add unit tests for the parser
|
||||
|
||||
**Example PR:**
|
||||
|
||||
- [feat(echo): Add Echo Support (trivy-db#528)](https://github.com/aquasecurity/trivy-db/pull/528)
|
||||
|
||||
### Step 3: Add OS/Ecosystem Support (Trivy)
|
||||
|
||||
Update [trivy] to support the new operating system or package ecosystem.
|
||||
|
||||
**Key tasks:**
|
||||
|
||||
- Add OS analyzer in `pkg/fanal/analyzer/os/` to detect the OS
|
||||
- Implement vulnerability detection logic if special handling is needed
|
||||
- Add integration tests with test data
|
||||
- Update documentation to include the new data source
|
||||
|
||||
**Example PR:**
|
||||
|
||||
- [feat(echo): Add Echo Support (trivy#8833)](https://github.com/aquasecurity/trivy/pull/8833)
|
||||
|
||||
## Complete Example: Echo OS Support
|
||||
|
||||
The Echo OS support was added through three coordinated PRs:
|
||||
|
||||
1. **vuln-list-update**: Fetches Echo advisories from `https://advisory.echohq.com/data.json`
|
||||
- PR: https://github.com/aquasecurity/vuln-list-update/pull/350
|
||||
2. **trivy-db**: Parses Echo advisories and stores them in the database
|
||||
- PR: https://github.com/aquasecurity/trivy-db/pull/528
|
||||
3. **Trivy**: Detects Echo OS and scans for vulnerabilities
|
||||
- PR: https://github.com/aquasecurity/trivy/pull/8833
|
||||
|
||||
## Testing Your Changes
|
||||
|
||||
### Test vuln-list-update
|
||||
|
||||
First, fetch all existing advisories (required for building the database):
|
||||
|
||||
```bash
|
||||
cd vuln-list-update
|
||||
go run main.go -vuln-list-dir /path/to/vuln-list
|
||||
```
|
||||
|
||||
Then, test your new data source by fetching only your target:
|
||||
|
||||
```bash
|
||||
go run main.go -target your-source -vuln-list-dir /path/to/vuln-list
|
||||
```
|
||||
|
||||
Verify that advisories are correctly saved in the vuln-list directory.
|
||||
|
||||
### Test trivy-db
|
||||
|
||||
```bash
|
||||
cd trivy-db
|
||||
make db-build CACHE_DIR=/path/to/cache
|
||||
```
|
||||
|
||||
Check that the database is built without errors and contains your advisories.
|
||||
|
||||
!!! note
|
||||
The `CACHE_DIR` should point to the parent directory of your vuln-list directory. For example, if your vuln-list is at `/tmp/test/vuln-list`, set `CACHE_DIR=/tmp/test`.
|
||||
|
||||
You can inspect the built database using BoltDB viewer tools like [boltwiz](https://github.com/Moniseeta/boltwiz):
|
||||
|
||||
```bash
|
||||
# Open the database
|
||||
boltwiz out/trivy.db
|
||||
```
|
||||
|
||||
This allows you to verify that your vulnerabilities are correctly stored in the database.
|
||||
|
||||
### Test Trivy
|
||||
|
||||
```bash
|
||||
# Build Trivy with your changes
|
||||
mage build
|
||||
|
||||
# Use your local database
|
||||
./trivy image --skip-db-update --cache-dir /path/to/cache your-test-image
|
||||
```
|
||||
|
||||
Verify that vulnerabilities from your new data source are detected correctly.
|
||||
|
||||
## Getting Help
|
||||
|
||||
If you have questions or need help:
|
||||
|
||||
1. Check existing data sources for reference implementations
|
||||
2. [Start a discussion](https://github.com/aquasecurity/trivy/discussions/new) in the Trivy repository
|
||||
|
||||
[vuln-list]: https://github.com/aquasecurity/vuln-list
|
||||
[vuln-list-update]: https://github.com/aquasecurity/vuln-list-update
|
||||
[trivy-db]: https://github.com/aquasecurity/trivy-db
|
||||
[trivy]: https://github.com/aquasecurity/trivy
|
||||
86
docs/community/contribute/vulnerability-database/overview.md
Normal file
86
docs/community/contribute/vulnerability-database/overview.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# Vulnerability Data Sources
|
||||
|
||||
This section explains how Trivy's vulnerability database works and how to contribute new advisory data sources.
|
||||
|
||||
## Overview
|
||||
|
||||
Trivy's vulnerability database is built through a multi-repository workflow involving three main repositories:
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
A[Advisory Sources] -->|vuln-list-update| B[vuln-list]
|
||||
B --> C["trivy-db<br/>(Trivy DB)"]
|
||||
C --> D["trivy<br/>(Trivy CLI)"]
|
||||
E[GitHub-managed<br/>Advisories] --> C
|
||||
```
|
||||
|
||||
### Workflow Steps
|
||||
|
||||
1. **Advisory Collection** ([vuln-list-update])
|
||||
- Fetch raw advisories from upstream sources
|
||||
- Store them in [vuln-list] repository
|
||||
- Run periodically via cron to keep advisories up-to-date
|
||||
- This step can be skipped if advisories are already managed in a Git repository (e.g., GitHub Security Advisories)
|
||||
|
||||
2. **Database Build** ([trivy-db])
|
||||
- Parse advisories from [vuln-list] or directly from Git-managed sources
|
||||
- Transform them into Trivy's database format
|
||||
- Publish the built database periodically via cron
|
||||
|
||||
3. **Database Consumption** ([trivy])
|
||||
- Download the latest vulnerability database at scan time
|
||||
- Use it to detect vulnerabilities in scan targets
|
||||
|
||||
## Why Store Advisories in vuln-list?
|
||||
|
||||
For data sources that are not already Git-managed, storing advisories in the [vuln-list] repository provides several benefits:
|
||||
|
||||
- **Transparency**: Easy to track changes and differences between advisory versions
|
||||
- **Web UI**: Browse advisories directly on GitHub with a user-friendly interface
|
||||
- **Stability**: Mitigate issues when upstream advisory servers are unstable or unavailable
|
||||
- **Shareability**: Provide stable URLs to reference specific advisories
|
||||
- **Data Quality**: Validate advisory data before committing to vuln-list, preventing malformed data or unexpected format changes from breaking Trivy DB
|
||||
- **Historical Data**: Preserve past advisories when upstream formats change
|
||||
|
||||
## Repository Overview
|
||||
|
||||
### [vuln-list-update]
|
||||
|
||||
This repository contains scripts that fetch advisories from various upstream sources. Each data source has its own package that handles:
|
||||
|
||||
- Fetching advisories from APIs or web sources
|
||||
- Validating the advisory format and data
|
||||
- Saving them to the [vuln-list] repository
|
||||
|
||||
### [vuln-list]
|
||||
|
||||
This repository serves as a data storage for raw advisories fetched by [vuln-list-update]. Key characteristics:
|
||||
|
||||
- Contains raw advisory data in JSON format
|
||||
- Updated automatically by [vuln-list-update] scripts via cron
|
||||
- **Not for manual contributions**: Direct pull requests to this repository are not accepted
|
||||
- Used as the source for [trivy-db] to build the vulnerability database
|
||||
|
||||
### [trivy-db]
|
||||
|
||||
This repository contains parsers that transform raw advisories into Trivy's database format. Each data source has its own vulnerability source handler that:
|
||||
|
||||
- Reads advisory files from [vuln-list] or directly from Git-managed sources (e.g., GitHub Security Advisories)
|
||||
- Maps advisory fields to Trivy's schema
|
||||
- Stores vulnerability information in the database
|
||||
|
||||
### [trivy]
|
||||
|
||||
The main Trivy repository contains:
|
||||
|
||||
- OS and package analyzers to detect what's installed
|
||||
- Vulnerability detection logic
|
||||
|
||||
## Next Steps
|
||||
|
||||
Ready to add a new vulnerability advisory source? See the [Add Vulnerability Advisory Source](add-vulnerability-source.md) guide for detailed steps.
|
||||
|
||||
[vuln-list]: https://github.com/aquasecurity/vuln-list
|
||||
[vuln-list-update]: https://github.com/aquasecurity/vuln-list-update
|
||||
[trivy-db]: https://github.com/aquasecurity/trivy-db
|
||||
[trivy]: https://github.com/aquasecurity/trivy
|
||||
@@ -232,6 +232,9 @@ nav:
|
||||
- Contribute Rego Checks:
|
||||
- Overview: community/contribute/checks/overview.md
|
||||
- Add Service Support: community/contribute/checks/service-support.md
|
||||
- Contribute Vulnerability Data Sources:
|
||||
- Overview: community/contribute/vulnerability-database/overview.md
|
||||
- Add Vulnerability Advisory Source: community/contribute/vulnerability-database/add-vulnerability-source.md
|
||||
- Maintainer:
|
||||
- PR Review: community/maintainer/pr-review.md
|
||||
- Release Flow: community/maintainer/release-flow.md
|
||||
|
||||
Reference in New Issue
Block a user