docs-drift has been failing on main since #118 #122

Problem

scripts/docs-drift.sh fails on main:

DRIFT: docs/content/reference/requirements.md documents '--no-robots' under 'ccrawl', which does not accept it
DRIFT: docs/content/reference/requirements.md documents '--robots' under 'ccrawl', which does not accept it

Both flags exist. crawl run has --no-robots and crawl fetch has --robots. The requirements page started talking about them in #118, and the page has no headings that name a command, so the script checks its flags against a hardcoded list of the commands the page covers. crawl fetch and crawl run were not on that list, so their flags had nothing to match against.

How it got in

docs-drift failed on PR #118 and the PR merged anyway. main has no branch protection, so gh pr merge --auto merges as soon as GitHub considers the PR mergeable, which has nothing to do with whether CI passed. Two runs of ci on main have been red since.

What to do

• Add crawl fetch and crawl run to the requirements page's scope list in the script.
• Separately, decide whether main should require the ci checks before a merge. That is a repository setting, not a code change, and it is the reason a red check merged.

Done when

• ./scripts/docs-drift.sh passes on main
• the two flags are checked against the commands that have them, so a real drift in that paragraph would still be caught