Skip to content

feat(seo): noindex collections and repo hubs, drop their sitemaps - #342

Merged
harlan-zw merged 2 commits into
mainfrom
feat/drop-authors-sources-sitemaps
Sep 30, 2026
Merged

harlan-zw merged 2 commits into
mainfrom
feat/drop-authors-sources-sitemaps

Conversation

@harlan-zw

@harlan-zw harlan-zw commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

❓ Type of change

  • 👌 Enhancement

📚 Description

Owner decision: "we don't need these indexed". Author profiles, collections, owner hubs and multi-Skill repository hubs now render noindex,follow. They stay live and linked. The authors and sources sitemaps are gone from the index and the module config.

A single-Skill repository hub is the Skill's own page. It stays index,follow when the trending admission rule admits it, and the skills sitemap still lists it. The sitemap equals the indexable set, and X-Robots-Tag repeats the meta, as in #322.

A hub that renders a Skill emits no robots tag of its own, so the Skill page decision is the only one on single-Skill hubs.

Loose end: the /gh/<owner> hubs were in sources.xml too, so they go noindex with the rest. Otherwise they would be indexable and unlisted.

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

Owner decision: author profiles, collections, owner hubs and multi-Skill repository hubs are not indexed. Their sitemaps go with them.
@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Sep 30, 2026
@harlan-github-agent

harlan-github-agent Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🤖 MERGED

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. Last updated: 2026-09-30 16:47 UTC.

GitHub merged this pull request.

  • Logged (55/100): The PR leaves an e2e assertion that demands the removed authors and sources sitemaps, contradicting the new test in the same file, so the e2e suite fails on every run. View code
    • Ask an agent to verify and fix this finding

Review completed. No finding above 80/100.

Selected findings run after merge and open separate pull requests.

One decision per page: a hub that renders a Skill emits no robots tag of its own.
@harlan-github-agent harlan-github-agent Bot removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Sep 30, 2026
@harlan-zw
harlan-zw merged commit b7380e3 into main Sep 30, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant