fix(public): make docs integration tests consumer-independent - #4016
Conversation
The public docs integration suite still read real on-disk guide content/counts in several places (tree grouping, the raw-markdown :slug.md test, the front-matter leak check). A consumer shipping its own guide at a globally-shared slug (welcome/quickstart) now deterministically wins that slug by design (#4011), so any assertion pinned to which real guide wins is a consumer-dependence bug. Split the file's first describe block into a shape/status-only block (no fixture needed, content is irrelevant) and a controlled-fixture block that fully replaces config.files.guides with self-authored guides (the #4014 pattern), restored in afterAll. The raw-markdown test now targets a fixture guide at a unique slug and asserts its own content. The slug-collision precedence block is untouched. Verified the invariant holds by running the suite with a synthetic extra guide file injected at the welcome/quickstart slugs from a fake non-core module — identical green result with or without it. Closes #4015
|
Warning Review limit reached
Next review available in: 44 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
WalkthroughThe integration tests now separate status and shape checks from content checks. Content checks use temporary guides configured through ChangesPublic docs integration tests
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #4016 +/- ##
=======================================
Coverage 93.54% 93.54%
=======================================
Files 170 170
Lines 5759 5759
Branches 1846 1846
=======================================
Hits 5387 5387
Misses 302 302
Partials 70 70
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@modules/public/tests/public.docs.integration.tests.js`:
- Around line 8-10: Move the unknown-slug assertion near the test using the
controlled guide fixture, rather than the configuration-independent tests.
Update that test to query a slug explicitly absent from the fixture list,
preserving the expected not-found response without relying on consumer-provided
guide configuration.
- Around line 82-85: Update the fixture written by fs.writeFileSync in the docs
integration test to begin with distinctive YAML front matter, then strengthen
the assertion around the response at the applicable test near lines 172-175 to
verify that marker is absent. Keep the existing documentation-content assertions
intact.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: d161165b-8187-4171-9425-09e9a2438f7c
📒 Files selected for processing (1)
modules/public/tests/public.docs.integration.tests.js
CodeRabbit review on #4016: the 404-for-unknown-slug test lived in the shape/status-only block and queried a hardcoded slug against whatever config.files.guides resolves to at boot. A consumer could in principle configure a guide at that exact slug, making the test consumer-dependent again. Move it into the controlled-fixture block (config.files.guides fully replaced there) and query a slug guaranteed absent from that fixture set.
Summary
modules/public/tests/public.docs.integration.tests.js— every test that read real on-disk guide content or counts now runs against a controlled, self-authored fixture guide set (config.files.guidesfully replaced for the duration of the block, restored inafterAll), or was scoped to a status/shape-only assertion where content is genuinely irrelevant. The raw-markdown:slug.mdtest now targets a fixture guide at a unique slug and asserts its own content.welcome/quickstart) deterministically wins that slug by design (an earlier, already-merged change). Any assertion in this suite that was pinned to which real guide wins a shared slug was a consumer-dependence bug by construction — it would fail in a consumer's own test run even though nothing was actually broken. This completes the same fix applied to one describe block in an earlier, already-merged PR, extended here to the rest of the file.Scope
public(docs)nonelowValidation
npm run lintnpm test(integration + unit, public-scoped)welcome/quickstartslugs from a fake non-core module (created under a throwaway module directory, removed immediately after): identical green result both times, confirming the suite no longer depends on which real guide wins a shared slug.Guardrails check
.env*,secrets/**, keys, tokens)Notes for reviewers
Summary by CodeRabbit