diff --git a/docs/community/contribute/vulnerability-database/add-vulnerability-source.md b/docs/community/contribute/vulnerability-database/add-vulnerability-source.md new file mode 100644 index 0000000000..c560ed3fc5 --- /dev/null +++ b/docs/community/contribute/vulnerability-database/add-vulnerability-source.md @@ -0,0 +1,144 @@ +# Add Vulnerability Advisory Source + +This guide walks through the process of adding a new vulnerability advisory source to Trivy. + +!!! info + For an overview of how Trivy's vulnerability database works, see the [Overview](overview.md) page. + +## Prerequisites + +Before starting, ensure you have: + +1. Identified the upstream advisory source and its API/format +2. Checked that the data source doesn't already exist in Trivy +3. Created a GitHub discussion or issue to discuss the addition + +## Required Changes + +To add a new vulnerability advisory source, you'll need to make changes across three repositories. Below we'll use the Echo OS support as an example. + +### Step 1: Add Fetcher Script (vuln-list-update) + +!!! note + Skip this step if your advisory source is already managed in a Git repository (e.g., GitHub, GitLab). + +Create a fetcher script in [vuln-list-update] to collect advisories from the upstream source. + +**Key tasks:** + +- Fetch advisories from the upstream API or source +- Validate the advisory format and data +- Save advisories as JSON files in the [vuln-list] directory structure +- **Store original data as-is where possible**: Avoid preprocessing or modifying advisory fields. Save the raw data exactly as provided by the upstream source (format conversion like YAML to JSON is acceptable for consistency) +- Include all necessary metadata (CVE ID, affected versions, severity, etc.) + +**Example PR:** + +- [feat(echo): Add Echo Support (vuln-list-update#350)](https://github.com/aquasecurity/vuln-list-update/pull/350) + +### Step 2: Add Parser (trivy-db) + +Create a parser in [trivy-db] to transform raw advisories into Trivy's database format. + +**Key tasks:** + +- Create a new vulnerability source in `pkg/vulnsrc/` +- Implement the advisory parsing logic +- Map advisory fields to Trivy's vulnerability schema +- Handle version ranges and affected packages correctly +- Store CVE mappings if available +- Add unit tests for the parser + +**Example PR:** + +- [feat(echo): Add Echo Support (trivy-db#528)](https://github.com/aquasecurity/trivy-db/pull/528) + +### Step 3: Add OS/Ecosystem Support (Trivy) + +Update [trivy] to support the new operating system or package ecosystem. + +**Key tasks:** + +- Add OS analyzer in `pkg/fanal/analyzer/os/` to detect the OS +- Implement vulnerability detection logic if special handling is needed +- Add integration tests with test data +- Update documentation to include the new data source + +**Example PR:** + +- [feat(echo): Add Echo Support (trivy#8833)](https://github.com/aquasecurity/trivy/pull/8833) + +## Complete Example: Echo OS Support + +The Echo OS support was added through three coordinated PRs: + +1. **vuln-list-update**: Fetches Echo advisories from `https://advisory.echohq.com/data.json` + - PR: https://github.com/aquasecurity/vuln-list-update/pull/350 +2. **trivy-db**: Parses Echo advisories and stores them in the database + - PR: https://github.com/aquasecurity/trivy-db/pull/528 +3. **Trivy**: Detects Echo OS and scans for vulnerabilities + - PR: https://github.com/aquasecurity/trivy/pull/8833 + +## Testing Your Changes + +### Test vuln-list-update + +First, fetch all existing advisories (required for building the database): + +```bash +cd vuln-list-update +go run main.go -vuln-list-dir /path/to/vuln-list +``` + +Then, test your new data source by fetching only your target: + +```bash +go run main.go -target your-source -vuln-list-dir /path/to/vuln-list +``` + +Verify that advisories are correctly saved in the vuln-list directory. + +### Test trivy-db + +```bash +cd trivy-db +make db-build CACHE_DIR=/path/to/cache +``` + +Check that the database is built without errors and contains your advisories. + +!!! note + The `CACHE_DIR` should point to the parent directory of your vuln-list directory. For example, if your vuln-list is at `/tmp/test/vuln-list`, set `CACHE_DIR=/tmp/test`. + +You can inspect the built database using BoltDB viewer tools like [boltwiz](https://github.com/Moniseeta/boltwiz): + +```bash +# Open the database +boltwiz out/trivy.db +``` + +This allows you to verify that your vulnerabilities are correctly stored in the database. + +### Test Trivy + +```bash +# Build Trivy with your changes +mage build + +# Use your local database +./trivy image --skip-db-update --cache-dir /path/to/cache your-test-image +``` + +Verify that vulnerabilities from your new data source are detected correctly. + +## Getting Help + +If you have questions or need help: + +1. Check existing data sources for reference implementations +2. [Start a discussion](https://github.com/aquasecurity/trivy/discussions/new) in the Trivy repository + +[vuln-list]: https://github.com/aquasecurity/vuln-list +[vuln-list-update]: https://github.com/aquasecurity/vuln-list-update +[trivy-db]: https://github.com/aquasecurity/trivy-db +[trivy]: https://github.com/aquasecurity/trivy \ No newline at end of file diff --git a/docs/community/contribute/vulnerability-database/overview.md b/docs/community/contribute/vulnerability-database/overview.md new file mode 100644 index 0000000000..4b4298621b --- /dev/null +++ b/docs/community/contribute/vulnerability-database/overview.md @@ -0,0 +1,86 @@ +# Vulnerability Data Sources + +This section explains how Trivy's vulnerability database works and how to contribute new advisory data sources. + +## Overview + +Trivy's vulnerability database is built through a multi-repository workflow involving three main repositories: + +```mermaid +graph LR + A[Advisory Sources] -->|vuln-list-update| B[vuln-list] + B --> C["trivy-db
(Trivy DB)"] + C --> D["trivy
(Trivy CLI)"] + E[GitHub-managed
Advisories] --> C +``` + +### Workflow Steps + +1. **Advisory Collection** ([vuln-list-update]) + - Fetch raw advisories from upstream sources + - Store them in [vuln-list] repository + - Run periodically via cron to keep advisories up-to-date + - This step can be skipped if advisories are already managed in a Git repository (e.g., GitHub Security Advisories) + +2. **Database Build** ([trivy-db]) + - Parse advisories from [vuln-list] or directly from Git-managed sources + - Transform them into Trivy's database format + - Publish the built database periodically via cron + +3. **Database Consumption** ([trivy]) + - Download the latest vulnerability database at scan time + - Use it to detect vulnerabilities in scan targets + +## Why Store Advisories in vuln-list? + +For data sources that are not already Git-managed, storing advisories in the [vuln-list] repository provides several benefits: + +- **Transparency**: Easy to track changes and differences between advisory versions +- **Web UI**: Browse advisories directly on GitHub with a user-friendly interface +- **Stability**: Mitigate issues when upstream advisory servers are unstable or unavailable +- **Shareability**: Provide stable URLs to reference specific advisories +- **Data Quality**: Validate advisory data before committing to vuln-list, preventing malformed data or unexpected format changes from breaking Trivy DB +- **Historical Data**: Preserve past advisories when upstream formats change + +## Repository Overview + +### [vuln-list-update] + +This repository contains scripts that fetch advisories from various upstream sources. Each data source has its own package that handles: + +- Fetching advisories from APIs or web sources +- Validating the advisory format and data +- Saving them to the [vuln-list] repository + +### [vuln-list] + +This repository serves as a data storage for raw advisories fetched by [vuln-list-update]. Key characteristics: + +- Contains raw advisory data in JSON format +- Updated automatically by [vuln-list-update] scripts via cron +- **Not for manual contributions**: Direct pull requests to this repository are not accepted +- Used as the source for [trivy-db] to build the vulnerability database + +### [trivy-db] + +This repository contains parsers that transform raw advisories into Trivy's database format. Each data source has its own vulnerability source handler that: + +- Reads advisory files from [vuln-list] or directly from Git-managed sources (e.g., GitHub Security Advisories) +- Maps advisory fields to Trivy's schema +- Stores vulnerability information in the database + +### [trivy] + +The main Trivy repository contains: + +- OS and package analyzers to detect what's installed +- Vulnerability detection logic + +## Next Steps + +Ready to add a new vulnerability advisory source? See the [Add Vulnerability Advisory Source](add-vulnerability-source.md) guide for detailed steps. + +[vuln-list]: https://github.com/aquasecurity/vuln-list +[vuln-list-update]: https://github.com/aquasecurity/vuln-list-update +[trivy-db]: https://github.com/aquasecurity/trivy-db +[trivy]: https://github.com/aquasecurity/trivy \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 4c75a0f63c..9d36659199 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -232,6 +232,9 @@ nav: - Contribute Rego Checks: - Overview: community/contribute/checks/overview.md - Add Service Support: community/contribute/checks/service-support.md + - Contribute Vulnerability Data Sources: + - Overview: community/contribute/vulnerability-database/overview.md + - Add Vulnerability Advisory Source: community/contribute/vulnerability-database/add-vulnerability-source.md - Maintainer: - PR Review: community/maintainer/pr-review.md - Release Flow: community/maintainer/release-flow.md