Skip to content

test(desktop): the Storybook smoke renders one of 22 palette × colour-scheme combinations #3942

Open

Description

@Astro-Han

Problem

THEME_PALETTES (packages/core/src/settings.ts:129-145) ships 11 palettes, each rendering in light and dark — 22 combinations reachable from 设置 → 外观.

scripts/storybook-visual-smoke.mjs:138 pins every story URL to globals=colorScheme:light and sets no palette, so the gate at .github/workflows/ci.yml:239 renders default × light only.

One story already works around this. packages/ui/stories/palette-matrix.stories.tsx applies data-maka-theme per block and renders all 11 palettes in a single story. Its own header states the reason:

The smoke's catalog pass renders wide/light/default only, so without this story ten palettes have no oracle whatsoever.

Three gaps remain on top of it:

  1. It is a human oracle, not an assertion. The story renders swatches; nothing fails when a value drifts.
  2. It covers 6 base tokens — --background, --foreground, --accent, --info, --success, --destructive (palette-matrix.stories.tsx:70-77). Derived tokens are out of frame. refactor(design-system): finish wiring the Maka theme to Astryx's token seams #3446 F6 measures --color-text-disabled at 2.52:1 light and 2.29:1 dark on transcript body copy, failing WCAG AA on the default palette. Whether the other 10 carry the same failure is unmeasured.
  3. The dark half is never rendered, so every dark-mode finding in refactor(design-system): finish wiring the Maka theme to Astryx's token seams #3446 is invisible to CI by construction.

maka-tokens.css:185 records a hand-check for one token ("weakest is tokyo-night at 4.78:1"). That is the current method: manual, per-token, unrepeatable.

Desired outcome

  • Parameterise storyUrl over colorScheme so the smoke runs both. Storybook already declares both globals (apps/desktop/.storybook/preview.tsx:83-118), so this is a URL change, not new infrastructure.
  • Add a per-text-node contrast assertion — no baseline, no allowlist, an accessibility correctness boundary. That is the shape test(ci): remove source-scanning governance gates #2794's objection leaves available.
  • Decide whether the palette dimension belongs in the gate or stays a human oracle. 11 × 2 over every story is likely not worth the CI time; over the palette matrix plus one reference screen it may be.

Overlaps #3446 slicing item 1, which already proposes the colour-scheme parameterisation and the contrast assertion. If that lands first, this narrows to the palette dimension.

AI use: Claude Code assisted with source and issue investigation; the analysis and conclusions are my own.

Activity

  1. Yx01-me commented on Aug 26, 2026

    @Yx01-me
    Contributor

    I'd like to work on this issue. Could you please assign it to me?

    For the Story coverage boundary, I propose:

    • Run every catalog Story under default × light and default × dark.
    • Run the Palette Matrix once per colour scheme; it already renders all 11 palettes in one Story.
    • Run one representative product Story, such as Native Conversation, across the full 11 palettes × 2 colour schemes matrix.
      This adds dark-mode coverage without multiplying the entire catalog by 22.

    For contrast, I plan to add a post-render audit after each Story's play() completes. It will inspect visible text nodes, resolve their computed foreground and effective background colours, and apply WCAG thresholds: 4.5:1 for normal text and 3:1 for large text. Only semantic exemptions such as genuinely inactive controls will be excluded—there will be no baseline or selector allowlist. Failures will report the Story, colour scheme, palette, text snippet, colours, measured ratio, and required ratio. Unit fixtures will cover colour calculation, transparent/alpha-composited ancestors, threshold selection, and exemptions.

  2. added theissue type on Aug 29, 2026
  3. Yx01-me commented on Sep 4, 2026

    @Yx01-me
    Contributor

    Progress update:

    #4042 completed the first stage: all Stories now cover light/dark, while Native Conversation covers all 11 palettes × 2 colour schemes.

    For the contrast assertion, I have a working @axe-core/playwright prototype, but need confirmation on the CI boundary:

    1. Should it cover all 410 render jobs, or only the 22 Native Conversation combinations? The full catalog currently contains substantial existing contrast failures.
    2. Should Axe incomplete results only be reported, or also fail CI? I recommend reporting them without blocking.
    3. Can we reuse test(desktop): expand Storybook smoke theme coverage #4042’s bounded palette coverage instead of expanding every Story to all 22 combinations?

    Once confirmed, I can finalize the second PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions