Skip to content

fix(pages): stop Claude reaching for an artifact when the user asked for a page - #7531

Merged
saddlepaddle merged 1 commit into
mainfrom
harshithmullapudi/super-2021-pages-over-artifacts
Sep 15, 2026
Merged

saddlepaddle merged 1 commit into
mainfrom
harshithmullapudi/super-2021-pages-over-artifacts

Conversation

@harshithmullapudi

@harshithmullapudi harshithmullapudi commented Sep 14, 2026 •

Copy link
Copy Markdown
Collaborator

Links

Summary

  • Adds a PreToolUse guard on Claude Code's Artifact tool that denies the first publish of a session inside a Superset terminal, pointing the agent at superset:page. The second call goes through untouched.
  • Teaches the three text surfaces we own (the page skill description and body, the pages_publish MCP tool description, the desktop page-agent prompt) to name artifacts explicitly, so the skill has a lexical reason to win.

Why / Context

From SUPER-2021: "Often i ask for a Superset page and it still generates a Claude artifact."

Artifact publishes a self-contained HTML document to a private URL, which is close enough to a page that the model picks it. It wins on prompt real estate: it is a one-step tool whose system-prompt description frames publishing as how a deliverable gets finished, while superset:page is a skill that must be invoked first. Before this PR nothing in the repo mentioned artifacts at all, so there was no signal pushing the other way.

The two are not interchangeable. A page is listed in the org, mints a version on every publish, and takes pinned comments that route back to an agent that can act on them. An artifact has none of that.

How It Works

The guard (packages/agent-setup/templates/artifact-guard.template.sh) stays completely silent unless all of:

Condition Why
SUPERSET_TERMINAL_ID/SUPERSET_TAB_ID set Hook configs are global; this must be inert in non-Superset Claude sessions
tool_name is Artifact
action is a publish list/comments/reply/read_db/write_db operate on an artifact that already exists
no url in tool_input A url means updating an artifact the user already owns
this session not yet nudged Marker file keyed by session_id, falling back to terminal id

Then it emits permissionDecision: "deny" with a reason naming superset:page and superset pages publish. SUPERSET_PAGES_NUDGE=off disables it outright.

Registration goes into ~/.claude/settings.json under PreToolUse with matcher Artifact, wrapped in the same SUPERSET_HOME_DIR guard the notify command uses.

Design Decisions

  • Why a separate script instead of another notify.sh event: the guard returns a permission decision on stdout. notify.sh is a fire-and-forget notifier whose only stdout is debug output, and mixing a decision into it would be fragile.
  • Why deny-once instead of a hard block: a hard block removes the choice from someone who genuinely wants a Claude artifact. Deny-once makes Pages the default while leaving a one-retry escape hatch that costs the agent nothing.
  • Why not text-only: a one-line skill description does not reliably out-rank a long imperative tool description in the system prompt. The wording helps; the hook is what actually changes the outcome.
  • Why cleanEntry had to change: it previously recognized managed entries by the notify script path alone. Without teaching it the guard path too, a re-merge would append a duplicate entry every boot and teardown would orphan it.

Manual QA Checklist

  • Provisioned into a temp SUPERSET_HOME_DIR, confirmed ~/.superset/hooks/artifact-guard.sh is written at mode 0755
  • Ran the exact registered command string against a real PreToolUse payload: first call → deny, second call → no output (tool proceeds)
  • PreToolUse entry merges as {"matcher":"Artifact", …} alongside the existing notify events
  • Re-merging over the hook's own output is idempotent (no duplicate Artifact entry)
  • Teardown removes the guard and leaves a user-authored PreToolUse hook in place
  • Guard is silent outside a Superset terminal, and with SUPERSET_PAGES_NUDGE in 0/off/false/no
  • Guard is silent for list/comments/reply/read_db/write_db and for a url-carrying update
  • Guard is silent for every other tool name
  • End-to-end in a real Superset terminal: ask an agent for a page, confirm it lands on Pages rather than an artifact

Testing

  • bun test in packages/agent-setup — 363 pass, 0 fail (10 new guard-script tests, 3 new registration tests)
  • bunx turbo typecheck --filter=@superset/desktop --filter=@superset/mcp --filter=@superset/agent-setup — clean
  • bunx biome check on the touched trees — clean
  • bun run check:plugins — passes
  • bun run check:i18n — no catalog churn (all added text is agent-facing, not interface copy)
  • bunx sherif — no issues

Known Limitations

  • Claude-only by construction. No other harness Superset launches has an Artifact tool, so Codex, Cursor, Droid et al. get the wording change and nothing else.
  • The plugin was not republished. plugins/superset/skills/page/SKILL.md changed without superset plugins publish superset --bump patch. There is no superset@* tag on origin, so check:plugins passes and the edited skill still ships via the bundled plugin dir — but if you want the marketplace release, that is a follow-up commit.
  • The guard's once-per-session marker lives in TMPDIR, so a machine that clears it mid-session gets one extra nudge. Harmless, and preferable to persisting state under ~/.superset.

Risks / Rollout

  • Risk: low, and scoped to Claude sessions launched from Superset terminals. Worst case is one denied tool call per session that the agent immediately retries.
  • Rollout: provisioning is declarative and runs at desktop/host boot; existing installs pick the hook up on next start.
  • Rollback: revert the commit. Next boot rewrites ~/.claude/settings.json without the PreToolUse entry, since teardown recognizes the guard command. SUPERSET_PAGES_NUDGE=off disables it without a deploy.

https://claude.ai/code/session_01R3SdsBKMTAniYr8U78UooQ


Summary by cubic

Stops Claude from reaching for a Claude artifact when the user asked for a Superset page. A PreToolUse hook now denies the first Artifact publish inside a Superset terminal and points the agent to the superset:page skill; the second call in the same session goes through untouched. The page skill, pages_publish tool description, and desktop agent prompt now name artifacts explicitly so the wording also pushes toward pages. The guard can be disabled with SUPERSET_PAGES_NUDGE=off.

Tracks SUPER-2021.

Notes

  • The guard only fires in Superset terminals, for publish actions without an existing url, once per session.
  • Existing installs pick up the hook on next boot; teardown removes it cleanly.
  • The edited SKILL.md ships via the bundled plugin dir; republishing the plugin to the marketplace is a follow-up.

Written for commit 658f943. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • New Features

    • Added a reminder that directs eligible artifact publishing to Superset Pages instead.
    • Prevents repeated reminders for the same session while allowing explicitly requested artifacts and opt-out behavior.
    • Pages guidance now highlights organization visibility, version history, pinned comments, and applicable content restrictions.
  • Bug Fixes

    • When the Pages skill is unavailable, the agent no longer substitutes a generic HTML workflow or Claude artifact and instead explains the required setup.
  • Tests

    • Added coverage for artifact publishing, session handling, opt-out behavior, and preserving existing configuration.

…for a page

Claude Code's Artifact tool publishes a self-contained HTML document to a
private URL, which is close enough to a Superset Page that the model picks it
for "make me a Superset page". It wins on prompt real estate: Artifact is a
one-step tool whose system-prompt description frames publishing as how a
deliverable gets finished, while superset:page is a skill that has to be
invoked first and, until now, never named the competitor. Nothing in the repo
mentioned artifacts at all, so there was no lexical reason for the model to
prefer the skill.

Text alone cannot reliably out-rank a system-prompt tool, so this pairs the
wording with a deterministic nudge. A new PreToolUse hook on Artifact denies
the first publish of a session with a reason pointing at superset:page, and
gets out of the way afterwards: the second call in the same session goes
through untouched, which is the escape hatch for someone who wanted a Claude
artifact. The guard only fires inside Superset terminals, only for a publish
(list/comments/reply/read_db/write_db and url-carrying updates pass straight
through), and SUPERSET_PAGES_NUDGE=off disables it entirely.

The guard is its own script rather than another notify.sh event because it
returns a permission decision on stdout, where notify.sh is a fire-and-forget
notifier whose output is debug-only. Claude's cleanEntry now recognizes both
scripts so re-merges do not duplicate the entry and teardown removes it while
leaving user PreToolUse hooks alone.

Claude-only by construction: no other harness Superset launches has an
Artifact tool, so the rest get the wording change and nothing else.

Claude-Session: https://claude.ai/code/session_01R3SdsBKMTAniYr8U78UooQ
@coderabbitai

coderabbitai Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 558f469e-5929-445f-86c1-c1b287c5b977

📥 Commits

Reviewing files that changed from the base of the PR and between 44c047f and 658f943.

📒 Files selected for processing (11)
  • apps/desktop/src/renderer/routes/_authenticated/_dashboard/pages/components/PagesView/hooks/useCreatePageWithAgent/constants.ts
  • packages/agent-setup/src/agent-setup.ts
  • packages/agent-setup/src/agent-wrappers-claude-codex-opencode.ts
  • packages/agent-setup/src/agent-wrappers-common.ts
  • packages/agent-setup/src/agent-wrappers.test.ts
  • packages/agent-setup/src/agent-wrappers.ts
  • packages/agent-setup/src/artifact-guard-hook.test.ts
  • packages/agent-setup/src/artifact-guard-hook.ts
  • packages/agent-setup/templates/artifact-guard.template.sh
  • packages/mcp/src/tools/pages/publish.ts
  • plugins/superset/skills/page/SKILL.md

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.


📝 Walkthrough

Walkthrough

Changes

The change adds a Superset Artifact guard hook. Shared setup provisions it, Claude registers it for Artifact pre-tool use, and tests cover filtering, session deduplication, idempotence, and cleanup. Pages documentation and agent guidance now define when Pages should replace Claude artifacts.

Artifact guard runtime and provisioning
packages/agent-setup/src/agent-wrappers-common.ts, packages/agent-setup/src/artifact-guard-hook.ts, packages/agent-setup/templates/artifact-guard.template.sh, packages/agent-setup/src/artifact-guard-hook.test.ts, packages/agent-setup/src/agent-setup.ts|The setup creates an executable guard script. The script filters Artifact publish events, tracks sessions, supports opt-out values, and emits a Pages-based denial when needed.|
|Claude hook integration
packages/agent-setup/src/agent-wrappers-claude-codex-opencode.ts, packages/agent-setup/src/agent-wrappers.ts, packages/agent-setup/src/agent-wrappers.test.ts|Claude receives a managed PreToolUse hook for the Artifact matcher. Merge operations remain idempotent, and teardown removes managed entries without removing user hooks.|
|Pages guidance and agent instructions
apps/desktop/.../useCreatePageWithAgent/constants.ts, packages/mcp/src/tools/pages/publish.ts, plugins/superset/skills/page/SKILL.md|Prompts, tool descriptions, and the page skill describe Pages as the preferred shared surface and document Artifact exceptions and the opt-out setting.|

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Suggested reviewers: kitenite

Sequence Diagram(s)

sequenceDiagram
  participant ClaudeCode
  participant ArtifactGuard
  participant SessionState
  participant SupersetPages
  ClaudeCode->>ArtifactGuard: submit Artifact publish event
  ArtifactGuard->>SessionState: check session marker
  ArtifactGuard-->>ClaudeCode: deny first publish when guard applies
  ClaudeCode->>SupersetPages: publish through Pages guidance
Loading

Merge Risk: ⚪ Minimal · up to 658f9

The new one-time reminder behaves as documented without blocking artifact updates or explicitly requested artifacts beyond the initial nudge.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.65% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 10 files. (1 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title uses conventional commit format and clearly summarizes the main change: preventing Claude from selecting an artifact when the user requests a Superset page.
Description check ✅ Passed The description provides the change summary, motivation, implementation details, testing results, limitations, and rollout information. It does not use the exact template headings or include the repos…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 17.65% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 10 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch harshithmullapudi/super-2021-pages-over-artifacts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

🧹 Preview Cleanup Complete

The following preview resources have been cleaned up:

  • ✅ Neon database branch

Thank you for your contribution! 🎉

@saddlepaddle
saddlepaddle merged commit a340acc into main Sep 15, 2026
20 checks passed
saddlepaddle pushed a commit that referenced this pull request Sep 23, 2026
* fix(agent-setup): write managed configs through symlinks

writeFileIfChanged writes a temp file and renames it onto the target.
rename never follows the final path component, so provisioning a user
config that is a symlink into a dotfiles repo unlinked the symlink and
left a regular file in its place. The read side does follow the link, so
the merged content was always correct and only the link was lost, which
is why nothing ever surfaced an error: from then on the home copy and the
repo copy drifted in silence.

Reported against ~/.claude/settings.json, where it recurred on every
restore: the repo copy lacks the artifact guard entry we started managing
in #7531, so the next provisioning run diffs, rewrites, and flattens the
link again. Every other managed config reached through this helper had
the same latent bug.

Resolve the target with realpath before reading or writing, and derive
the temp path from the resolved target so a link pointing at another
filesystem still renames instead of failing EXDEV. A dangling link keeps
the old behaviour and is claimed, matching how linkSharedDir treats one.

* fix(agent-setup): only claim the path when the link is unusable

The realpath fallback caught every error, so a target we could not resolve
for reasons that have nothing to do with links — an unreadable parent
directory, a parent that is a regular file — fell through to a write against
the literal path. Narrow it to ENOENT and ELOOP, the two cases where there is
genuinely no inode to write through and the path is ours to claim, and let
anything else reach the caller.

Raised in review on #7696: ELOOP was the case worth naming, since a cyclic
link resolves to nothing and should be replaced like a dangling one rather
than abort provisioning.

* fix(host-service): seed agent trust through symlinked stores

The trust seeder is the other tmp-file + rename writer for a user-owned
config, and it ran unfixed: every workspace creation appends a
[projects."…"] block to $CODEX_HOME/config.toml and ~/.claude.json, so a
config.toml symlinked into a dotfiles repo was flattened far more often
than the settings.json case #7696 fixes.

Share the resolution step as resolveWriteTarget instead of copying it, so
the two writers cannot drift on which errors mean "claim the path".

* fix(agent-setup): name both paths when a linked config is read-only

Resolving the link turned one silent failure into a loud one: a
~/.claude/settings.json managed by Nix home-manager points into
/nix/store, so the write now throws EACCES where it used to flatten the
link. Failing is the right half of that trade — home-manager refuses its
next activation once a managed file is a regular file — but a bare
EACCES on a temp path tells the user nothing.

Wrap EACCES/EPERM/EROFS with both paths, so the warning agent-setup logs
says which link points where and why hooks were not installed.

* fix(agent-setup): name both paths when a linked config cannot be read

The read is the other half of the same failure. On Linux realpath happily
resolves a target the process cannot read, so a link into a store that
denies reads threw a bare EACCES naming only the store path — the write
side already said which link pointed where. macOS fails earlier, inside
realpath, with an error that names the link itself, which is why the test
is skipped there.

Chmod stays best-effort on purpose: when the content already matches,
the config is correct and a store that refuses chmod would otherwise fail
every boot with nothing for the user to fix.

* fix(agent-setup): key the linked-path error on lstat, not path equality

realpath canonicalizes more than links: a relative path, a path through
.., or anything under /tmp on macOS comes back different from what the
caller passed, so a plain EACCES on an ordinary file claimed the file
"links to" its own canonical path. Ask lstat instead, and only when an
error is already in hand.

Both permission tests also needed a Windows guard: it enforces neither
0o444 nor 0o222, and process.getuid is undefined there, so the old skip
condition let them run and fail.

* fix(workspace-fs): write through a symlinked file instead of replacing it

Every file an agent or the editor writes goes through writeAtomically, so
a symlinked file inside a repo — a CLAUDE.md pointing at a shared copy, a
config linked out of a dotfiles repo — was replaced by a regular file on
the first save. This is the same bug #7696 fixed for managed agent
configs, on the path that touches user repos.

Resolving cannot escape the workspace: writeFile already runs
assertRealpathWithinRoot, which rejects a link whose target leaves the
root, so the resolved path is inside it by the time we rename. A test
pins that refusal alongside the new behaviour.

The precondition read moves from lstat to stat for the same reason.
readFile reports the target's revision, so comparing it against the
link's own lstat would report a conflict on every conditional write once
the link survives — the editor's read-modify-write loop would never
succeed on a symlinked file.

* fix(cli,host-service): write the remaining user configs through symlinks

Sweep of the writers left with the tmp-file + rename shape after #7696:
the CLI's ~/.superset/config.json and app-state.json, the host service's
auth config (SUPERSET_AUTH_CONFIG_PATH), and the default-account pointer.
Each one replaced a symlink with a regular file, so a config kept in a
dotfiles repo stopped receiving writes — silently, because the write
itself succeeded. The auth config is the one that bites hardest: a
refreshed token would land on a copy the CLI no longer reads.

All four now resolve through agent-setup's resolveWriteTarget, and derive
their temp path from the resolved target so a link across filesystems
still renames.

Left alone deliberately: session-share.ts manages links by design (it
creates them and checks for them); the self-updaters replace a binary at
a path, which is the point; Electron userData state, the pty handoff
snapshot in tmpdir, and the shell-ready evidence cache are app-internal
files nobody links, and are regenerated when lost.

* fix(host-service): keep the terminal env path off the agent-setup barrel

CI caught it: src/terminal/*.node-test.ts run under node
--experimental-strip-types, and agent-setup's index re-exports with
extensionless relative specifiers that Node's ESM resolver rejects. My
resolveWriteTarget import pulled that barrel onto the terminal
env-resolution path through default-account.ts, which is exactly what its
"must stay free of the agent-setup surface" comment warns about.

So default-account.ts mirrors the helper, as it already does for
resolveSupersetHomeDir and resolveAmbientCodexHome. Everything else
imports it from a new @superset/agent-setup/write-file-if-changed
subpath, which Node resolves because that module pulls in only node:fs.

This branch was previously deployed

1 inactive deployment
preview — 658f943e Deployed Sep 14, 2026 by harshithmullapudi via Deploy API #9833
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants