Skip to content

Eliminate HTML from skill reference files - #1825

Merged
max-sixty merged 4 commits into
mainfrom
skills-xml
Mar 31, 2026
Merged

max-sixty merged 4 commits into
mainfrom
skills-xml

Conversation

@max-sixty

Copy link
Copy Markdown
Owner

Skill reference files are consumed by LLMs, not browsers. They contained ~400 HTML tags (<b><span class=g>Commands:</span></b>, &#39;, etc.) that are noise for LLM consumers.

The root cause: skill files were derived from web-formatted docs via transform_docs_for_skill(), which stripped Zola shortcodes but passed HTML through. The HTML came from two pipeline stages that only serve web rendering: post_process_for_html() (CLI markers → HTML spans in prose) and convert_command_reference_to_html() (ANSI → HTML in reference blocks).

For command pages (7 files, ~350 occurrences): adds --help-page --plain flag that skips web-specific transforms (post_process_for_html, ANSI color codes, terminal shortcodes, demo GIF expansion). The sync test routes command pages through this flag instead of reading from HTML-formatted docs. The --plain output reuses the same assembly logic (prose + reference block + subdoc expansion) but produces clean markdown.

For non-command pages (5 files, ~50 occurrences): the HTML was in hand-written terminal shortcode bodies (styled CLI output demos). Changed the terminal body fallback in transform_docs_for_skill from passing through raw HTML to calling the existing strip_html() function. Also strips rawcode shortcodes. No new text processors added — reuses existing infrastructure.

Key files: src/help.rs (parameterized with plain: bool), tests/integration_tests/readme_sync.rs (new generate_skill_from_help + finalize_skill_content shared helper), src/commands/config/state.rs (fix pre-existing render_markdown_table → render_data_table migration bug from ec6f6f2).

This was written by Claude Code on behalf of maximilian

max-sixty and others added 4 commits March 30, 2026 14:57
Skill files are consumed by LLMs, not browsers. They contained ~400 HTML
tags (`<b><span class=g>Commands:</span></b>`, `&#39;`, etc.) that are
noise for LLM consumers.

Root cause: skill files were derived from web-formatted docs, which add
HTML for rendering. The fix generates skill content from a pre-HTML stage
instead of trying to reverse HTML after the fact.

For command pages (7 files): add `--help-page --plain` flag that skips
web-specific transforms (post_process_for_html, ANSI color codes,
terminal shortcodes, demo GIF expansion). The sync test routes command
pages through this flag instead of reading from HTML-formatted docs.

For non-command pages (5 files): add character-level `strip_html_tags`
(no regex on XML) that strips tags outside code fences and decodes HTML
entities.

Also converts the HTML `<table class="cmd-compare">` in worktrunk.md to
a markdown table, which auto-syncs to README.md.

Co-Authored-By: Claude <noreply@anthropic.com>
Resolve merge conflict in config.md (take plain-text format, add new
`vars` subcommand from main). Fix pre-existing bug where
`render_markdown_table` call in state.rs wasn't migrated to
`render_data_table` during the extraction in ec6f6f2.

Co-Authored-By: Claude <noreply@anthropic.com>
Revert worktrunk.md table conversion — the README needs the HTML table
with <pre> tags for GitHub rendering. Instead, handle non-command page
HTML in transform_docs_for_skill by:

1. Reusing existing strip_html() for terminal shortcode bodies that
   don't have a cmd parameter (styled CLI output demos)
2. Stripping rawcode shortcodes ({% rawcode() %}X{% end %} → X)

Removes the standalone strip_html_tags character-level parser and all
its supporting functions — the existing strip_html() already handles
tag stripping and entity decoding.

Co-Authored-By: Claude <noreply@anthropic.com>
# Conflicts:
#	src/commands/config/state.rs
@max-sixty
max-sixty merged commit dc6a88b into main Mar 31, 2026
24 checks passed
@max-sixty
max-sixty deleted the skills-xml branch March 31, 2026 02:04
max-sixty added a commit that referenced this pull request Apr 1, 2026
Skill reference files are consumed by LLMs, not browsers. They contained
~400 HTML tags (`<b><span class=g>Commands:</span></b>`, `&#39;`, etc.)
that are noise for LLM consumers.

The root cause: skill files were derived from web-formatted docs via
`transform_docs_for_skill()`, which stripped Zola shortcodes but passed
HTML through. The HTML came from two pipeline stages that only serve web
rendering: `post_process_for_html()` (CLI markers → HTML spans in prose)
and `convert_command_reference_to_html()` (ANSI → HTML in reference
blocks).

**For command pages** (7 files, ~350 occurrences): adds `--help-page
--plain` flag that skips web-specific transforms
(`post_process_for_html`, ANSI color codes, terminal shortcodes, demo
GIF expansion). The sync test routes command pages through this flag
instead of reading from HTML-formatted docs. The `--plain` output reuses
the same assembly logic (prose + reference block + subdoc expansion) but
produces clean markdown.

**For non-command pages** (5 files, ~50 occurrences): the HTML was in
hand-written terminal shortcode bodies (styled CLI output demos).
Changed the terminal body fallback in `transform_docs_for_skill` from
passing through raw HTML to calling the existing `strip_html()`
function. Also strips `rawcode` shortcodes. No new text processors added
— reuses existing infrastructure.

Key files: `src/help.rs` (parameterized with `plain: bool`),
`tests/integration_tests/readme_sync.rs` (new `generate_skill_from_help`
+ `finalize_skill_content` shared helper),
`src/commands/config/state.rs` (fix pre-existing `render_markdown_table`
→ `render_data_table` migration bug from ec6f6f2).

> _This was written by Claude Code on behalf of maximilian_

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants