Skip to content

fix(ci): upload both cluster logs from the container benchmark jobs - #2460

Merged
andygrove merged 1 commit into
apache:mainfrom
amitvijapur:fix/ci-cluster-log-artifact-container-paths
Sep 17, 2026
Merged

andygrove merged 1 commit into
apache:mainfrom
amitvijapur:fix/ci-cluster-log-artifact-container-paths

Conversation

@amitvijapur

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #2147.

Rationale for this change

The Upload cluster logs on failure step in the three container benchmark jobs (tpch.yml, tpcds.yml, h2o.yml) lists the two log files as a multi-line path. In a container job the runner translates host paths to container paths for action inputs with a single prefix replacement on the whole input string (ContainerInfo.TranslateToContainerPath, applied to every environment value in StepHost.cs before docker exec; node action inputs arrive there as INPUT_*). A two-line input starts with the host path once, so only its first line is rewritten. actions/runner#1174 is this defect, open since 2021.

upload-artifact then gets one path that exists inside the container and one that does not. From a failed PR leg on the current form of the step (run 35050467408):

The least common ancestor is /. This will be the root directory of the artifact
No files were found with the provided path: /__w/_temp/scheduler.log
/home/runner/work/_temp/executor.log. No artifacts will be uploaded.

The least common ancestor of /__w/... and /home/runner/... is /, which is the line quoted in the issue, and it is why the one file that did upload in run 29749959848 sat under __w/_temp/ inside the artifact.

What changes are included in this PR?

The two-line path becomes one glob, ${{ runner.temp }}/*.log, in all three workflows, with a two-line comment saying why it is a glob so it does not get tidied back into two lines. A single line is rewritten whole, and with one search path the artifact root is /__w/_temp, so both files land at the top level of the artifact.

{scheduler,executor}.log was not an option: @actions/glob constructs minimatch with nobrace: true.

Within this repo's own workflow and ci/ code, the only writer of .log files into runner.temp is ci/scripts/ballista_cluster.sh (scheduler.log, executor.log, via CLUSTER_DIR).

Are these changes tested?

Yes, with a throwaway workflow on my fork that runs in the same amd64/rust container, writes both files to $CLUSTER_DIR, and uploads them twice: once with the current two-line path, once with the glob. In https://github.com/amitvijapur/datafusion-ballista/actions/runs/35105812662 the two-line step logs there will be 1 file uploaded and its artifact holds only __w/_temp/scheduler.log; the glob step logs there will be 2 files uploaded and its artifact holds scheduler.log and executor.log at the root. The step's own output also shows the env translation at work: the step header prints CLUSTER_DIR: /home/runner/work/_temp while the shell inside the container sees CLUSTER_DIR=/__w/_temp.

Are there any user-facing changes?

No. CI only. Failure artifacts from these three jobs now contain executor.log, and the files sit at the artifact root instead of under __w/_temp/.

Written with AI assistance (Claude Code). The mechanism was checked against the runner and toolkit sources and the run logs quoted above, and the repro was run on my fork.

In a container job the runner rewrites only the leading host path of a
multi-line `path:` input (actions/runner#1174), so `executor.log` was
never uploaded. A single glob is rewritten as one path.

Closes apache#2147

@andygrove andygrove left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @amitvijapur

@andygrove
andygrove merged commit 0c764a5 into apache:main Sep 17, 2026
12 checks passed
@amitvijapur
amitvijapur deleted the fix/ci-cluster-log-artifact-container-paths branch September 20, 2026 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: TPC-H SF10 failure artifact is missing executor.log

2 participants