Skip to content

fix(analyzer): filter license boilerplate from EA3 static findings (#312) - #328

Open
rodboev wants to merge 2 commits into
NVIDIA:mainfrom
rodboev:pr/static-runner-license-ea3
Open

fix(analyzer): filter license boilerplate from EA3 static findings (#312)#328
rodboev wants to merge 2 commits into
NVIDIA:mainfrom
rodboev:pr/static-runner-license-ea3

Conversation

@rodboev

@rodboev rodboev commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Static-only scans currently report EA3 Scope Creep findings from Apache-2.0 boilerplate in LICENSE, COPYING, and NOTICE files. This change keeps those files in the scan inventory while filtering the EA3 false positive only where the matched line is recognized license boilerplate in a text-like legal-basename file.

Closes #312

Root cause

The static runner applies EA3 to every cached text-like component. Apache-2.0 contains the phrase not limited to, which matches EA3 even though license text is not skill instruction content. The default LLM path can discard these matches later, but --no-llm reports them directly and sends avoidable findings through the analysis pipeline.

A filename-only suppression cannot be the whole fix: skill filenames are attacker-controlled, so a file named LICENSE.md could hide an excessive-agency instruction from EA3. Suppression therefore also requires the matched line to be recognized license boilerplate, which closes that bypass.

Diff Notes

  • Add delimiter-aware, case-insensitive LICENSE, COPYING, and NOTICE basename handling in static_runner.py.
  • Add content-based license-boilerplate validation: an EA3 finding is suppressed only when the matched line is a canonical license phrase for a recognized license family (Apache-2.0, MIT, BSD).
  • Report EA3 for any legal-basename file whose matched line is not recognized boilerplate; an instruction moved into a license-named file stays detectable.
  • Preserve non-EA3 scanning, inspection-ledger completion, direct analyzer behavior, SKILL.md detection, and ordinary prose detection.
  • Add production-path regressions for legal filename families, filename boundaries, ledger accounting, and direct EA3 behavior, plus the adversarial regression test_license_named_file_with_non_boilerplate_content_reports_ea3.

The suppression behavior follows the reproduction documented in issue 312, including the clarification that the false positive is exposed by --no-llm scans.

Scope

The filter applies only to EA3 matches whose line is recognized license boilerplate inside a text-like legal basename file. License-named files with non-boilerplate content are still reported. Other findings, non-license files, inventory, and report behavior remain unchanged.

Verification

  • python -m pytest tests/nodes/analyzers/test_static_patterns.py tests/nodes/analyzers/test_binary_and_pe3_filtering.py tests/unit/test_patterns_new.py - 393 passed
  • python -m pytest tests/nodes/analyzers/test_static_patterns.py -k "non_boilerplate or embedded_instruction or boilerplate" - adversarial regressions, 11 passed
  • uv run ruff check src/ tests/ - All checks passed
  • uv run ruff format --check src/ tests/ - 157 files already formatted
  • skillspector scan --no-llm --format json - EA3 location.file is ["SKILL.md"] and no LICENSE

@rng1995 rng1995 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Automated SkillSpector Review]

Requesting changes. This suppresses every EA3 finding in any text-like file whose basename resembles LICENSE, COPYING, or NOTICE, without verifying that the matched text is license boilerplate. Skill files remain untrusted regardless of their name, so malicious excessive-agency instructions can be moved into LICENSE.md and bypass EA3 entirely. Please scope suppression to recognized boilerplate content (or otherwise validate legal-file content) and add an adversarial regression showing that non-license instructions in a license-named file remain detectable.

for module in pattern_modules:
raw = module.analyze(content=content, file_path=path, file_type=file_type)
for af in raw:
if af.rule_id == "EA3" and _is_license_basename(path, file_type):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking security issue: this drops every EA3 match based only on the attacker-controlled filename. A skill can place an excessive-agency instruction in LICENSE.md and evade the rule. Please require recognized license-boilerplate content (or another strong legal-file validation) before suppression, and add a regression that preserves EA3 for malicious/non-license content under a license-like basename.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review. The filename-only gate was too broad, and you're right that a legal-looking name is attacker-controlled. Suppressing on name alone meant a file named LICENSE.md could carry an instruction that EA3 would otherwise report. I've scoped suppression to content.

  1. static_runner.py now only drops an EA3 finding when all three hold: the file is text-like with a legal basename (the existing _is_license_basename), the file is recognized as a standard license family, and the specific matched line is a canonical license phrase. The new _is_license_boilerplate_line handles the last two at the gate in _scan_path.
  2. Recognized families are Apache-2.0, MIT, and BSD, detected by distinctive whole-file markers. The canonical-line check covers the EA3 phrase as it appears in each family's standard text: including but not limited to and not limited to compiled object code in Apache, and the shared but not limited to phrase in MIT and BSD full texts. The issue-312 false positive stays fixed.
  3. Any EA3 match whose line is not a canonical license phrase is still reported. A LICENSE.md that holds only an instruction, or an instruction line slipped into an otherwise genuine Apache file, still produces an EA3 finding.
  4. The earlier tests that used the issue phrase as fake license content now use a recognizable Apache-2.0 block. The guard the review asked for is test_license_named_file_with_non_boilerplate_content_reports_ea3, with the embedded-instruction and ledger variants sitting next to it.

The net guarantee is specific: an EA3 match is suppressed only when its line is a recognized license-boilerplate phrase in a license-named file. Instructions under a license-like name are reported. One caveat stays on the table: a line that itself embeds a canonical license phrase (for example an instruction sentence containing "including but not limited to") is still suppressed; that is the boundary of the recognized-boilerplate check.

NVIDIA#312)

Only suppress an EA3 finding on a text-like legal basename when the matched
line is recognized license boilerplate content, so instructions smuggled into
license-named files stay reported.

Signed-off-by: Rod Boev <rodboev@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EA3 flags Apache-2.0 LICENSE boilerplate as scope creep (fires on 814/817 skills in one corpus)

2 participants