Skip to content

ci(jest): shards fail with "SyntaxError: Unexpected end of JSON input" from event.json race in parallel group #127566

Closed

Description

@billyvg

Summary

Since #126667 moved the Jest shard setup into a parallel: group, shards sometimes fail before jest runs with:

SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at new Context (/home/runner/work/_actions/actions/download-artifact/d3f86a106a0bac45b974a628896c90dbdf5c8093/dist/index.js:4845:37)

This is a race in the GitHub Actions runner: every step start truncates and rewrites the shared $GITHUB_EVENT_PATH file. Any action that loads @actions/github parses that file at startup, so it crashes if a sibling step in the group starts at the same moment. The bug is reported upstream as actions/runner#4641; the fix PRs (actions/runner#4643, actions/runner#4675) are still open with no maintainer response.

We can fix it on our side without giving up the parallel: speedup.

Evidence

Failures

I checked every frontend.yml run that ended in failure between 2026-09-24 and 2026-10-15:

Date (UTC) Job Failed step Action that crashed
2026-10-09 09:59 Jest (2, 8) (PR) Download jest-balance.json dawidd6/action-download-artifact
2026-10-13 17:23 Jest (5, 8) (push to master) Download jest-test-files.json actions/download-artifact
2026-10-15 03:01 Jest (2, 8) (push to master) Download jest-test-files.json actions/download-artifact
  • The same error appears zero times from Sep 24 to Oct 7, when these steps ran one after another. All three cases are after perf(ci): Fetch jest shard dependencies in parallel #126667 merged (2026-10-07 15:46 UTC).
  • Treat 3 as a lower bound. I only scanned runs whose final result was a failure, so runs that someone re-ran green aren't counted.

In each case the action crashes inside new Context(), 0.2–0.9s after it starts, while other steps in the group are still starting. Timeline from job 116223369053:

03:01:04.295  Run actions/cache                          <- "jest transform cache" starts
03:01:04.317  Run actions/download-artifact              <- "Download jest-test-files.json" starts, node boots
03:01:04.318  Download action repository 'actions/setup-node'   <- setup-node-pnpm composite is starting
03:01:04.495  SyntaxError: Unexpected end of JSON input  <- download-artifact reads an empty event.json
                  at new Context (.../actions/download-artifact/.../dist/index.js:4845:37)
03:01:04.618  Run dawidd6/action-download-artifact       <- "Download jest-balance.json" starts

In job 113768151765 the crash is in dawidd6, through the same code path:

SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at new Context (.../dawidd6/action-download-artifact/ac66b43.../node_modules/@actions/artifact/node_modules/@actions/github/lib/context.js:15:37)

Mechanism (runner v2.337.0, which our jobs run on)

  • ActionRunner.RunAsync calls WriteWebhookPayload() for every step start. That includes run: steps and each inner step of a composite action.
  • WriteWebhookPayload does File.WriteAllText(<temp>/_github_workflow/event.json, …), which truncates the file and then writes it. The file is shared by all steps in the job.
  • @actions/github creates context = new Context() when the module loads. The constructor does JSON.parse(readFileSync(process.env.GITHUB_EVENT_PATH)), so it throws if it reads during the truncation window.

Which steps in our group read the event file

Step in the group Reads $GITHUB_EVENT_PATH at startup?
actions/setup-node (first step of .github/actions/setup-node-pnpm) yes: the entry module requires @actions/github at the top level (dist/setup/index.js)
actions/download-artifact yes: dist/index.js#L4845
dawidd6/action-download-artifact yes: through @actions/github in its node_modules
pnpm/action-setup, actions/cache (restore and save), bash run: steps, gh CLI no

Post steps aren't affected. The runner waits for background steps to finish and then runs post steps one at a time.

Ruled out: giving each step its own copy of the event file

Setting env: GITHUB_EVENT_PATH: <copy> on a step has no effect. The runner applies the step's env: first (ActionRunner.cs#L251) and then overwrites all GITHUB_* variables from the github context (NodeScriptActionHandler.cs#L42-L52). This is also actions/runner#701.

Proposed solutions

1. Recommended: keep parallel:, but run every event-file reader before it

- uses: actions/checkout@…

# Actions that load @actions/github (setup-node, download-artifact, checkout, github-script, …)
# parse $GITHUB_EVENT_PATH at startup, and the runner truncates and rewrites that file every
# time any step starts (actions/runner#4641). Keep them out of the parallel group below.
- uses: actions/setup-node@1e60f620b9541d16bece96c5465dc8ee9832be0b # v4
  with:
    node-version-file: .node-version

- parallel:
    - uses: ./.github/actions/setup-node-pnpm
      with:
        setup-node: 'false'   # new input, defaults to 'true' so the other 19 callers don't change

    - name: jest transform cache
      uses: actions/cache@…   # unchanged

    - name: Download jest artifacts
      env:
        GH_TOKEN: ${{ github.token }}
        GH_REPO: ${{ github.repository }}
      run: ./.github/workflows/scripts/download-jest-artifacts.sh
  • In .github/actions/setup-node-pnpm/action.yml: add a setup-node input (default 'true'), and add if: inputs.setup-node == 'true' to the actions/setup-node step.
  • Replace both download actions with a single gh step, which never reads the event file. The two downloads also get retries this way; see "Related" below.

Script sketch (not yet tested):

#!/usr/bin/env bash
# Download jest-test-files.json (uploaded by "Configure Jest" in this run) and
# jest-balance.json (latest successful jest-balance.yml run on master).
set -euo pipefail

retry() {
  local attempt
  for attempt in 1 2 3; do
    "$@" && return 0
    echo "::warning::Attempt $attempt failed: $*" >&2
    sleep $((attempt * 5))
  done
  return 1
}

# Extract to a fresh dir so a retry never trips over a partial earlier attempt.
download() {  # <run-id> <artifact-name> <dest-dir>
  local tmp
  tmp=$(mktemp -d)
  gh run download "$1" --name "$2" --dir "$tmp" && mkdir -p "$3" && cp -R "$tmp"/. "$3"/
}

retry download "$GITHUB_RUN_ID" jest-test-files .

balance_run=$(retry gh run list --workflow jest-balance.yml --branch master --status success \
  --limit 1 --json databaseId --jq '.[0].databaseId // empty')
if [[ -z "$balance_run" ]]; then
  # Every shard sees the same result here, so they all fall back to splitting by name.
  echo "::warning::No successful jest-balance.yml run on master; shards will split by file name" >&2
  exit 0
fi
retry download "$balance_run" jest-balance.json tests/js/test-balancer
  • Downloading jest-balance.json must not silently fail. If one shard is missing the file, createJestConfig.ts makes that shard split tests by filename while the other shards split by duration. Some tests would then run twice and others not at all, with nothing failing.
  • Mid-run downloads work. v4 artifacts are available through the REST API as soon as the upload finishes, so gh run download "$GITHUB_RUN_ID" can fetch jest-test-files while the run is still going.
  • Cost: roughly nothing.

2. Smaller change: replace only the two download actions with the gh step

This leaves actions/setup-node as the only reader in the group. It starts 0.4–2.5s after the group begins, inside the composite, and is usually past the point where the other branches start. But in job 115437592082, setup-node and dawidd6 started 24ms apart. So this option depends on timing and doesn't guarantee the race is gone.

3. Revert #126667

This also works, but loses about 9s per shard.

Upstream

Upvote or ping actions/runner#4643 or actions/runner#4675. Once one of them merges, the readers can move back into the group.

Related, but a different cause

The same two download steps fail for other reasons too. Between 2026-10-05 and 2026-10-15:

  • Unknown error: {}: 18 Jest jobs. The balance zip download gets an HTTP error from GitHub's artifact storage, and @octokit/request 8.4.1 inside dawidd6 v11 hides the response body. This started Oct 5, before perf(ci): Fetch jest shard dependencies in parallel #126667.
  • Server Error: 15 Jest jobs, during GitHub's 2026-10-12 Actions incident.

Neither is caused by the race. The retries in option 1 should cover both, and gh will print the real HTTP status when it does fail.

Activity

  1. linear-code commented on Oct 15, 2026

    @linear-code
  2. added a commit that references this issue on Oct 16, 2026
    cf5f0ed
  3. added a commit that references this issue on Oct 16, 2026
    ca8a733
  4. added 2 commits that reference this issue on Oct 19, 2026
    d8eb4ec
    65720da
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Component: CIContinuous Integration pipeline (GitHub Actions)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions