Commit Graph

171 Commits

Author SHA1 Message Date
Mike Hunhoff
1572dd87ed lint: add WARN for regex features that contain unescaped dot (#2635)
* lint: add WARN for regex features that contain unescaped dot

* refactor comments

* update CHANGELOG

* address PR feedback

* Update scripts/lint.py

Co-authored-by: Willi Ballenthin <wballenthin@google.com>

* Update scripts/lint.py

Co-authored-by: Willi Ballenthin <wballenthin@google.com>

---------

Co-authored-by: Willi Ballenthin <wballenthin@google.com>
2025-03-18 15:05:57 -06:00
Willi Ballenthin
6d19226ee9 rules: scopes can now have subscope blocks with same scope (#2584) 2025-02-03 19:54:05 +01:00
vibhatsu
2798d605bc add lint for duplicate feature under a statement (#2573)
* add lint for duplicate feature under a statement

* add support for more scopes

* fix format for duplicate feature lint

* fix false positives for duplicate features lint

* remove unused code and comments

Signed-off-by: vibhatsu <maulikbarot2915@gmail.com>

* refactor duplicate feature lint to use yaml parser

Signed-off-by: vibhatsu <maulikbarot2915@gmail.com>

* update CHANGELOG

Signed-off-by: vibhatsu <maulikbarot2915@gmail.com>

* clarify for using rule definition

Signed-off-by: vibhatsu <maulikbarot2915@gmail.com>

* update CHANGELOG

Signed-off-by: vibhatsu <maulikbarot2915@gmail.com>

* refactor duplicate feature lint to improve key generation and tracking of line numbers

Signed-off-by: vibhatsu <maulikbarot2915@gmail.com>

---------

Signed-off-by: vibhatsu <maulikbarot2915@gmail.com>
2025-01-31 10:05:53 +01:00
Willi Ballenthin
712e35c6f7 feat: add lint to validate rule dependency scope compatibility
closes #2124
2025-01-29 18:53:30 +01:00
Willi Ballenthin
cdc1cb7afd rename "sequence" scope to "span of calls" scope
pep8

fix ref

update submodules

update testfiles submodule

duplicate variable
2025-01-29 02:25:06 -07:00
Willi Ballenthin
b06fea130c dynamic: add sequence scope
addresses discussion in
https://github.com/mandiant/capa-rules/discussions/951

pep8

sequence: add test showing multiple sequences overlapping a single event
2025-01-29 02:25:06 -07:00
Willi Ballenthin
8d17319128 capabilities: use dataclasses to represent complicated return types
foo
2025-01-29 02:25:06 -07:00
Ana Maria Martinez Gomez
3cd97ae9f2 [copyright + license] Fix headers
Replace the header from source code files using the following script:
```Python
for dir_path, dir_names, file_names in os.walk("capa"):
    for file_name in file_names:
        # header are only in `.py` and `.toml` files
        if file_name[-3:] not in (".py", "oml"):
            continue
        file_path = f"{dir_path}/{file_name}"
        f = open(file_path, "rb+")
        content = f.read()
        m = re.search(OLD_HEADER, content)
        if not m:
            continue
        print(f"{file_path}: {m.group('year')}")
        content = content.replace(m.group(0), NEW_HEADER % m.group("year"))
        f.seek(0)
        f.write(content)
```

Some files had the copyright headers inside a `"""` comment and needed
manual changes before applying the script. `hook-vivisect.py` and
`pyinstaller.spec` didn't include the license in the header and also
needed manual changes.

The old header had the confusing sentence `All rights reserved`, which
does not make sense for an open source license. Replace the header by
the default Google header that corrects this issue and keep capa
consistent with other Google projects.

Adapt the linter to work with the new header.

Replace also the copyright text in the `web/public/index.html` file for
consistency.
2025-01-15 08:52:42 -07:00
mr-tz
2987eeb0ac update type annotations
tmp
2024-10-22 09:38:25 +00:00
Fariss
51a4eb46b8 replace tqdm, termcolor, tabulate with rich (#2374)
* logging: use rich handler for logging

* tqdm: remove unneeded redirecting_print_to_tqdm function

* tqdm: introduce `CapaProgressBar` rich `Progress` bar

* tqdm: replace tqdm with rich Progress bar

* tqdm: remove tqdm dependency

* termcolor: replace termcolor and update `scripts/`

* tests: update `test_render.py` to use rich.console.Console

* termcolor: remove termcolor dependency

* capa.render.utils: add `write` & `writeln` methods to subclass `Console`

* update markup util functions to use fmt strings

* tests: update `test_render.py` to use `capa.render.utils.Console`

* replace kwarg `end=""` with `write` and `writeln` methods

* tabulate: replace tabulate with `rich.table`

* tabulate: remove `tabulate` and its dependency `wcwidth`

* logging: handle logging in `capa.main`

* logging: set up logging in `capa.main`

this commit sets up logging in `capa.main` and uses a shared
`log_console` in `capa.helpers` for logging purposes

* changelog: replace packages with rich

* remove entry from pyinstaller and unneeded progress.update call

* update requirements.txt

* scripts: use `capa.helpers.log_console` in `CapaProgressBar`

* logging: configure root logger to use `RichHandler`

* remove unused import `inspect`
2024-09-27 09:34:21 +02:00
N0stalgikow
0eb4291b25 Updating copyright across all files based on when it was first introduced. (#2027)
* updating copyright, back to the date of origin of file

* updating regex to account for linter violation
2024-03-13 14:04:53 +01:00
Aayush Goel
49231366f1 Handles circular dependencies while getting rules and dependencies (#2014)
* Remove test for scope "unspecified"

* raise error on circular dependency

* test for circular dependency
2024-03-06 11:39:21 +01:00
Moritz
2c93c5fc83 lint: get backend from format (#1964)
* get backend from format

* add lint.py script test

* create FakeArgs object

* adjust EOL handling in lints

---------

Co-authored-by: Willi Ballenthin <wballenthin@google.com>
2024-02-01 11:33:16 +01:00
mr-tz
66c2f07ca8 remove BaseException usage 2024-01-31 11:32:00 +01:00
dependabot[bot]
ba044a980f build(deps-dev): bump black from 23.12.1 to 24.1.1 (#1955)
* build(deps-dev): bump black from 23.12.1 to 24.1.1

Bumps [black](https://github.com/psf/black) from 23.12.1 to 24.1.1.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/23.12.1...24.1.1)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* black 24.1.1 formatting

* update flake config to match black 24.1.1 format

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
Co-authored-by: mr-tz <moritz.raabe@mandiant.com>
2024-01-31 11:18:54 +01:00
Willi Ballenthin
c3301d3b3f refactor main to for ease of integration (#1948)
* main: split main into a bunch of "main routines"

[wip] since there are a few references to BinExport2
that are in progress elsewhre. Next commit will remove them.

* main: remove references to wip BinExport2 code

* changelog

* main: rename first position argument "input_file"

closes #1946

* main: linters

* main: move rule-related routines to capa.rules

ref #1821

* main: extract routines to capa.loader module

closes #1821

* add loader module

* loader: learn to load freeze format

* freeze: use new cli arg handling

* Update capa/loader.py

Co-authored-by: Moritz <mr-tz@users.noreply.github.com>

* main: remove duplicate documentation

* main: add doc about where some functions live

* scripts: migrate to new main wrapper helper functions

* scripts: port to main routines

* main: better handle auto-detection of backend

* scripts: migrate bulk-process to main wrappers

* scripts: migrate scripts to main wrappers

* main: rename *_from_args to *_from_cli

* changelog

* cache-ruleset: remove duplication

* main: fix tag handling

* cache-ruleset: fix cli args

* cache-ruleset: fix special rule cli handling

* scripts: fix type bytes

* main: remove old TODO message

* loader: fix references to binja extractor

---------

Co-authored-by: Moritz <mr-tz@users.noreply.github.com>
2024-01-29 13:59:05 +01:00
Yacine Elhamer
96fb204d9d move capa.features.capabilities to capa.capabilities, and update scripts 2023-10-20 09:54:24 +02:00
Willi Ballenthin
1aac4a1a69 mypy 2023-10-17 14:42:58 +00:00
Willi Ballenthin
44d05f9498 dynamic: fix some tests 2023-10-17 11:41:40 +00:00
Yacine Elhamer
5730e5515f lint.py: update recommendation messages 2023-08-23 01:42:22 +02:00
Yacine Elhamer
901ba551bc lint.py: fix boolean statement 2023-08-23 01:41:44 +02:00
Yacine Elhamer
77b3fadf79 lint.py: add 'unsupported' keyword 2023-08-23 01:39:14 +02:00
Yacine Elhamer
44fc3357d1 initial commit 2023-08-23 01:32:01 +02:00
Willi Ballenthin
c1fbb27d73 Merge branch 'master' into dynamic-feature-extraction 2023-08-10 13:21:49 +00:00
mr-tz
9660f1e5ab add default backend 2023-08-07 14:00:30 +02:00
Willi Ballenthin
85f151303a merge 2023-08-07 08:40:03 +00:00
Willi Ballenthin
a538a7bbab linter: skip native API check for more UEFI routines 2023-08-02 12:55:22 +00:00
Willi Ballenthin
ab5c8b1129 linter: skip native API check for NtEnumerateSystemEnvironmentValuesEx 2023-08-02 12:49:17 +00:00
Willi Ballenthin
04fbcbbbd3 linter: skip native API check for NtProtectVirtualMemory
closes #1675
2023-08-02 12:36:42 +00:00
Yacine Elhamer
e38e56ccf6 Merge remote-tracking branch 'parentrepo/dynamic-feature-extraction' into sync-1657 2023-07-20 09:33:48 +01:00
Willi Ballenthin
c86ab51210 fix copyright headers everywhere 2023-07-13 05:03:33 +02:00
Willi Ballenthin
65e8300145 introduce flake8-simplify 2023-07-12 11:40:44 +02:00
Yacine Elhamer
4ee38cbe29 fix linting issues 2023-07-11 14:52:04 +01:00
Yacine Elhamer
f86ecfe446 Merge remote-tracking branch 'parentrepo/dynamic-feature-extraction' into analysis-flavor 2023-07-11 10:43:31 +01:00
Aayush Goel
1baa7a5e4b flake8 checks resolved 2023-07-11 02:30:09 +05:30
Aayush Goel
ef39bc3c3a Merged Changes from PR #1591 2023-07-11 01:14:38 +05:30
Aayush Goel
8e346cb411 Merge branch 'Aayush-Goel-04/Issue#1534' of https://github.com/Aayush-Goel-04/capa into Aayush-Goel-04/Issue#1534 2023-07-11 00:59:21 +05:30
Yacine Elhamer
917dd8b0db Update scripts/lint.py
Co-authored-by: Willi Ballenthin <willi.ballenthin@gmail.com>
2023-07-10 15:58:17 +01:00
Willi Ballenthin
7fe738e28f introduce flake8-no-implicit-concat linter 2023-07-09 22:18:01 +02:00
Willi Ballenthin
54203f3be9 introduce flake8-logging-format linter 2023-07-09 22:11:46 +02:00
Aayush Goel
673af45c55 Update args.sample type to Path and str vs as_posix comparisons 2023-07-09 16:02:28 +05:30
Aayush Goel
e0ed8c6e04 Resolved the suggestions. 2023-07-08 13:51:41 +05:30
Aayush Goel
ef9b0737a8 Merge branch 'master' into Aayush-Goel-04/Issue#1534 2023-07-07 12:05:57 +05:30
Aayush Goel
14924174c5 convert str(path) usage to path.as_posix() to get str format of Path
Update fixtures.py
2023-07-07 12:03:05 +05:30
Aayush Goel
edeb458b33 some more changes 2023-07-07 12:03:05 +05:30
Willi Ballenthin
13a8e252f0 introduce flake8-comprehensions 2023-07-06 20:04:27 +02:00
Willi Ballenthin
982dc46623 add flake8-bugbear linter 2023-07-06 19:30:51 +02:00
Yacine Elhamer
0c56291e4a update linter 2023-07-06 17:50:57 +01:00
Yacine Elhamer
c916e3b07f update the linter 2023-07-06 17:27:45 +01:00
Willi Ballenthin
47074fd129 fix ruff issues 2023-07-06 17:49:40 +02:00