[cli-consistency] CLI Consistency Issues - 2026-09-14 #60843

Summary

Automated inspection of the gh aw CLI help output (/tmp/gh-aw/agent/all-help.txt, 481 per-command help files) and comparison against docs/src/content/docs/setup/cli.md. 3 issues found: 1 high, 1 medium, 1 low. No typos or grammar issues were found in help text.

Severity breakdown: 🔴 High: 1 · 🟡 Medium: 1 · ⚪ Low: 1

Findings

🔴 High — gh aw logs --artifacts default described as "usage" but actual default is "info"

Affected: gh aw logs command help and long description; docs/src/content/docs/setup/cli.md (multiple locations)

The --artifacts flag's default value was changed from usage to info, but the surrounding help text and doc prose were not updated to match, so the CLI's own help output is self-contradictory.

Exact quoted CLI output (gh aw logs --help):

By default, only the compact usage artifact is downloaded (token usage, run metadata).
Use --artifacts all to download all artifacts, or specify individual sets such as
--artifacts agent,firewall to fetch only what you need.
...
  # Artifact selection (default: usage only - the compact conclusion artifact)
...
      --artifacts strings               Artifact sets to download (default: info — compact workflow metadata). Use 'all' for everything, or comma-separate sets. Valid sets: activation, agent, all, detection, evals, experiment, firewall, github-api, graders, info, mcp, usage (default [info])

Note the flag description explicitly says (default: info ...) and (default [info]), while the prose two lines above and the example comment still say "usage" / "usage only".

Source: pkg/cli/logs_command.go:82 and pkg/cli/logs_command.go:151 still contain the stale "usage" wording (identified via grep -rn "compact usage artifact\|usage only - the compact conclusion" pkg/cli/logs_command.go).

docs/src/content/docs/setup/cli.md line 515 also states:

`logs` defaults `--artifacts` to `usage` for faster, compact downloads.

and line 552 states:

Unlike `gh aw logs`, which defaults to `usage`, `audit` defaults to `all` for comprehensive analysis.

Both of these should say info, not usage.

Expected vs actual:

• Expected: help text and docs consistently state the default artifact set is info.
• Actual: the flag's own --help metadata says info/[info], but adjacent prose (in the same --help output) and the docs say usage.

Suggested fix: Update pkg/cli/logs_command.go line 82 comment and line 151 description to say "info" instead of "usage", and update docs/src/content/docs/setup/cli.md lines 515 and 552 accordingly.

Priority: High — directly misleading, appears in the command's own --help output, and is likely to confuse users about which artifacts are downloaded by default.

🟡 Medium — docs/src/content/docs/setup/cli.md audit artifact-set list is missing info

Affected: docs/src/content/docs/setup/cli.md line 552; gh aw audit --help

Actual CLI output (gh aw audit --help):

--artifacts strings   Artifact sets to download (default: all — comprehensive artifacts required for analysis). Valid sets: activation, agent, all, detection, evals, experiment, firewall, github-api, graders, info, mcp, usage

Docs text (line 552):

Valid sets include `activation`, `agent`, `all`, `detection`, `evals`, `experiment`, `firewall`, `github-api`, `graders`, `mcp`, and `usage`.

Expected vs actual: The docs' enumerated list of valid artifact sets omits info, which is a real, currently valid value accepted by both gh aw logs --artifacts and gh aw audit --artifacts.

Suggested fix: Add info to the enumerated list in the docs (activation, agent, all, detection, evals, experiment, firewall, github-api, graders, info, mcp, usage).

Priority: Medium — incomplete but not actively wrong; users trying --artifacts info would still have it work, they'd just not know about it from the docs.

⚪ Low — gh aw edit and gh aw json-schema commands are undocumented in docs/src/content/docs/setup/cli.md

Affected: docs/src/content/docs/setup/cli.md; gh aw edit --help, gh aw json-schema --help

Both edit and json-schema are real, currently-registered top-level commands (confirmed via gh aw --help):

  edit             Experimental: edit workflow frontmatter and recompile
  ...
  json-schema      Generate JSON Schemas for structured command output

docs/src/content/docs/setup/cli.md has no #### edit or `#### `json-schema section (confirmed via grep -n "edit\|json-schema" across the file — only incidental substring matches like "editing" and "read/edit/search tools" appear, and one indirect mention of gh aw json-schema logs-jsonl in the --cached-jsonl prose, but no dedicated section with usage/options).

Expected vs actual: Every top-level command listed in gh aw --help should have a corresponding documented section (as compile, run, add, etc. do).

Suggested fix: Add #### edit and `#### `json-schema sections to the CLI docs, following the same format as neighboring commands (short description, example usage block, **Options:** line). edit is explicitly marked "Experimental" in its help text, so the docs entry should carry that same caveat.

Priority: Low — both are lower-traffic/experimental or utility commands, but the gap reduces discoverability for users browsing the docs rather than --help.

Inspection Metadata

• Commands inspected: All top-level commands and subcommands from the pre-collected /tmp/gh-aw/agent/all-help.txt (25,497 lines) and 481 individual files under /tmp/gh-aw/agent/help-output/, including: compile, run, add, add-wizard, validate, lint, format, fix, list, status, logs, audit, trial, deploy, update, upgrade, secrets (+ set, bootstrap), env (+ get, update), mcp (+ add, inspect, list, list-tools), project (+ new), pr (+ transfer), checks, health, forecast, outcomes (+ history), experiments (+ list, analyze), models, graders (+ run), domains, hash-frontmatter, json-schema, completion (+ install, uninstall), doctor, init, new, remove, disable, enable, edit, mcp-server, version.
• Date: 2026-09-14
• Method: Manual textual diff between pre-collected --help output (treated as authoritative source of truth) and docs/src/content/docs/setup/cli.md (1106 lines), plus a source-code grep confirmation (pkg/cli/logs_command.go) for the stale "usage" default wording, and a full-file typo/grammar scan (no issues found).

Generated by ✅ CLI Consistency Checker · copilot · auto · 119.3 AIC · ⌖ 7.01 AIC · ⊞ 7.4K · ◷

• expires on Sep 16, 2026, 5:33 AM UTC-08:00