Skip to content

fix: labeling - workflows fight over PR labels (broken labeler rules, non-canonical type:documentation, stale payloads) #3545

Closed

Description

@eleshar

Raised from Ash's comment on #3525 (#3525 (comment)). On #3525, github-actions[bot] made about 60 label changes between 05:37 and 08:29 UTC on 2026-09-24. Each of the 24 pushes re-runs the labelling workflows, and they disagree. Verified from the PR timeline, workflow job times and logs, and the code on develop.

1. area:testing (and every file-based label) is added, then removed

  • PR Template & Label Routing adds area:testing, because docs(specs): spec 016 standardised Claude Code cloud environment #3525 changes tests/js/claude-cloud-environment-docs.test.js. Example: added at 07:08:50; the routing job ran 07:08:25–07:08:54.
  • Labeling • Unified / File/branch labels (PRs) (actions/labeler v7.0.0, bf12e9b, sync-labels: true) removes it. Example: removed at 07:09:13; the job ran 07:08:58–07:09:15.
  • Root cause: .github/labeler.yml writes changed-files: as a mapping:
    area:testing:
      - changed-files:
          any-glob-to-any-file:
            - tests/**/*
    In labeler v7.0.0, src/changedFiles.ts toChangedFilesMatchConfig starts with if (!config['changed-files'] || !config['changed-files'].length) return {};. A mapping has no .length, so the rule is dropped silently and never matches, and sync-labels then removes the label. All 15 changed-files rules use this form (area:assets, area:block-editor, area:ci, area:documentation, area:jest, area:pagespeed, area:phpunit, area:playwright, area:scripts, area:testing, area:theme, area:woocommerce, lang:css, lang:js, lang:php), so no file-based labeller rule has ever matched. The 22 branch rules work.
  • Fix: use the list form (- changed-files: / - any-glob-to-any-file: [...]), or let a single workflow own area labels.

2. type:documentation is added, but labels.yml only defines type:docs

  • scripts/agents/labeling.agent.js maps docs, doc, documentation, readme, guide and the docs/ and doc/ branch prefixes to type:documentation (L54–L58, L94–L95). Log from the 07:08 push: Detected type from branch prefix 'docs/': type:documentation; 1 added.
  • Other literals in the agent that are not in labels.yml: type:accessibility, type:dependencies, meta:changelog.
  • PR template frontmatter labels not in labels.yml: type:documentation (pr_docs.md), type:ai-ops, meta:needs-review. On docs(specs): spec 016 standardised Claude Code cloud environment #3525, routing applied type:docs, not the template list.
  • Fix: take type labels from labels.yml, or a mapping shared with the router, instead of hard-coded strings.

3. More than one type: label, and it flip-flops on every push

In one agent run (07:05 push; its current labels included type:docs and type:documentation):

  1. enforceOneHotLabels keeps the first type: label and removes type:documentation (removed at 07:08:02; the agent job ran 07:07:21–07:08:05).
  2. standardizeLabelsOnItem then tries to remove type:documentation again as non-canonical, gets a 404 and aborts: Label standardization failed: Label does not exist.
  3. On the next push the payload no longer lists type:documentation, so branch detection adds it again (07:10:36).

The agent also reads labels from the event payload, which is stale by the time a queued run starts. That is how type:chore was added at 20:19:03 on 2026-09-23, seconds after routing had set type:docs.

  • Fix: read the item's current labels from the API before enforcing one-hot rules or adding defaults, and do not add a label that the same run's standardisation would remove.

Notes

4. Issue type labels: the agent guesses by keyword and ignores GitHub's issue type (added 2026-09-24)

Found on #3554, which was created from Linear. Verified from the issue timeline, the Labeling • Unified run 35995809300 log, and scripts/agents/labeling.agent.js on develop.

Time (UTC) Actor Change
11:54:45 Linear app Issue created with GitHub issue type Chore (correct)
11:55:00–11:55:23 github-actions[bot] (labelling agent) Added status:needs-triage, priority:normal, type:task and type:bug
12:03:59 Linear sync Removed priority:normal, type:bug and type:task; added priority:minor, area:core and type:chore

Linear's labels agree with the issue type because it writes both from its own record. Our agent produced two conflicting type: labels, and neither matched the issue type:

  • type:task came from applyDefaultType ([label-enforcer] No type label found on #3554, applying default: type:task).
  • type:bug came from detectIssueTypeFromContent (Detected type from body keyword 'issue': type:bug). KEYWORD_TYPE_MAP is checked by plain substring, and the first match wins. issue: "type:bug" (L48) comes before chore (L73), so nearly any issue that uses the word "issue" becomes a bug. Other short keys misfire the same way: ci matches "decision" and "specific", fix matches "prefix", doc matches "document".
  • Both steps work from the same starting payload labels, so neither sees the other's label, and one-hot enforcement never sees both.
  • The agent never reads the native issue type: there is no type/issue_type reference in labeling.agent.js.

Fix

  • Source of truth: when an issue has a GitHub issue type, derive its type:* label from that type (Chore → type:chore, Bug → type:bug, Task → type:task, Feature → type:feature, and so on), and do not add a default or a keyword-guessed type. The same applies on edited, when the issue type changes.
  • Keyword fallback (no issue type set): match whole words, not substrings; drop over-broad keys such as issue; and prefer explicit template signals (for example a "Chore Summary" heading) over body keywords.
  • One decision per run: decide one type:* label per run (from issue type, template, branch or keyword, in that order), then apply it, instead of a default step and a keyword step each adding one.
  • Canonical names: keep every mapped label in labels.yml (see item 2).

Also noted on #3554

Linked Stories/Tasks/PRs

Milestones & Timeline


Definition of Ready (DoR)

  • Bug is reproducible and clearly described
  • Steps to reproduce written (timelines and logs above)
  • Environment details provided (develop, actions/labeler v7.0.0)
  • Screenshots/logs included (if possible)
  • Linked to existing issues/PRs (if any)
  • Estimate added (if relevant)
  • Ready for triage

Definition of Done (DoD)

Metadata

Metadata

Assignees

No one assigned

    Type

    Fields

    Priority

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions