You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Run the Regression Harness on a Local Branch Image - #1200
The regression harness always ran docker pull docker.io/ptr727/plexcleaner:<tag>. So a branch could only get a regression run after it merged and a develop build published it. #1192 and #1193 each need a run before they merge, so this lets the harness run a locally built branch image.
Local image. A tag holding a colon names a local image reference, such as plexcleaner:local, because no Docker tag can contain a colon. It runs without a pull, records "Channel": "local" in buildinfo.json, and skips the plugin builds. A plugin builds from the release tag its image version names, and a branch build has no release tag. find_run matches Channel exactly, so a tool picking the newest develop run never picks a local one.
Published references.docker.io/ptr727/plexcleaner:X and ptr727/plexcleaner:X run as the tag X, so they keep their channel and results directory. A digest reference, [docker.io/]ptr727/plexcleaner@sha256:<digest>, runs as published and pulls by digest.
Version label. The label names the results directory under the root-run mkdir, cp, and chown -R, so it must be a plain name, ^[0-9A-Za-z][0-9A-Za-z._+-]*$. That also keeps a quote out of the buildinfo.json heredoc. A local image's label must also be a commit hash, ^[0-9a-f]{7,40}$. That refuses the Dockerfile default 1.0.0.0, which every unlabelled build would share, and any published version, which always holds a dot. So a local run never lands in a published build's results directory. The README has you commit first, then build with --build-arg LABEL_VERSION=$(git rev-parse --short=10 HEAD). The results directory is then the commit, and git rev-parse <version>^{commit} resolves it the same way it resolves a published run's tag.
Function signatures. The internal functions drop their unused Tag parameter in favor of one ImageRef, and their signature comments change to match. The PR carries the comments label for that reason.
Verification:
shellcheck and shfmt -d are clean, and so are prose_lint.py --diff origin/develop --allow-comments and markdownlint-cli2. --help and the unknown-argument refusal are unchanged.
A real run needs root and the ZFS corpus, so the harness was run against stubbed docker, zfs, git, dotnet, id, and filesystem writes, with HostConfig redirected to a scratch directory. A local image skips the pull and the plugins, and records Channel: local and its own reference. Both published reference spellings pull docker.io/ptr727/plexcleaner:<tag> and record the tag as their channel. A digest reference pulls by digest. A missing local image, a local image labelled with the default or a published version, and a label such as ../.. each exit 1.
Findings from the local review
introduced, fixed in e63ca3b: a digest-pinned published reference ran as local. Copilot and Qodo raised it too.
Fixed in 259f734, from CodeRabbit: a local image labelled with a published version would overwrite that version's results directory.
pre-existing, declined: an image the harness runs can plant a symlink in a results directory through the existing read-write --volume "$HostConfig:$ConfigPath:rw" mount. A later root cp or redirect then writes through it. The mount, the cp, and the redirect all predate this change. Running an image already trusts its code with read-write media and results mounts, so a malicious image is outside the harness's threat model, per GOVERNANCE.md "Trust Boundaries and Hardening Effort".
Declined: a race where the local tag is rebuilt during the seconds between WriteBuildInfo and the docker run calls. That would need the maintainer to retag an image on his own host mid-run, which the same section rules out guarding.
Local review: 5 agent-skill passes at the frontier tier, the last over content digest 021d935ac4c5.
A tag holding a colon names a local image, which the harness runs without a pull,
so a branch gets a regression run before any published image carries it.
The harness refuses a local image still carrying the default version label, and
records the run under a local channel so no develop run picker selects it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 46db3595-9606-4f1a-95ec-7d72f9a9a674
Commits
Reviewing files that changed from the base of the PR and between e63ca3b and 259f734.
Files selected for processing: 2
RegressionTests/README.md
RegressionTests/regression-test.sh
No actionable comments were generated in the recent review.
Walkthrough
The regression harness now accepts local Docker image references as well as published tags and digests. It skips pulls and plugin builds for local images, validates their version label, records their channel and image reference, and uses the resolved image for regression and coverage launches.
Changes: Local image regression runs
Layer / File(s)
Summary
Resolve image and run metadata OPERATIONS.md, RegressionTests/README.md, RegressionTests/regression-test.sh
The documentation describes local image references. The harness resolves the image and channel, checks the image version, and records the channel and image reference in build metadata.
Run with the resolved image RegressionTests/regression-test.sh
Regression, settings, schema, and coverage launches use the resolved image reference. The harness skips pulls and plugin builds for local images.
sequenceDiagram
participant RegressionTest as regression-test.sh
participant Docker
participant SetLaunch
participant BuildPlugin
RegressionTest->>Docker: Pull ImageRef when Channel is not local
RegressionTest->>Docker: Inspect ImageRef
RegressionTest->>SetLaunch: Launch ImageRef for regression or coverage
alt Channel is local
RegressionTest-->>BuildPlugin: Skip plugin build
else Published channel
RegressionTest->>BuildPlugin: Build plugin for the version
end
Loading
Possibly related issues
ptr727/PlexCleaner#1177 — The issue says the harness accepts only published image tags, which blocks regression testing of an unmerged branch. This change adds local image support.
Merge Risk: Low · previous review e63ca
Local-image runs are usable, but a local image carrying a published version label can overwrite existing regression results. Merge with awareness of this bounded risk; use the documented commit-hash labels.
Pre-merge checks: 5
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title "Run the Regression Harness on a Local Branch Image" matches the main change: the harness now accepts a locally built branch image (colon-containing reference), skips pulling and plugin buil…
Docstring Coverage
Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title "Run the Regression Harness on a Local Branch Image" matches the main change: the harness now accepts a locally built branch image (colon-containing reference), skips pulling and plugin builds, and records Channel: local. The title is specific and concise, and the changes in RegressionTests/regression-test.sh and README.md support it.
Full details: Docstring Coverage
Explanation
Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 7 functions across 1 files. (1 skipped: 1 unsupported.)
Finishing Touches
Generate docstrings
Commit to this branch
Create a new PR
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 6623c9f1-dae0-440f-89ff-02aff956a016
Commits
Reviewing files that changed from the base of the PR and between 7a723a1 and e63ca3b.
Files selected for processing: 2
RegressionTests/README.md
RegressionTests/regression-test.sh
Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.
A recorded local strict-review pass covers head 7a723a1e443bf83c10e2ce16beac6eec3a0915b0, the content this pull request carries at that commit against develop, and it recorded 1 finding.
Run regression tests against locally built branch images
✨ Enhancement📝 Documentation🕐 20-40 Minutes
AI Description
• Run branch images without pulling, enabling regression checks before publication.
• Keep local results separate from published channels and reject images with the default version
label.
• Preserve published tag behavior and document commit-labeled builds for traceable results.
Diagram
graph TD
A["Image argument"] --> B{"Local image?"}
B -->|published| C["Pull image"] --> D{"Valid version?"}
B -->|local| D
D -->|no| E["Reject run"]
D -->|yes| F["Record buildinfo"] --> G{"Local channel?"}
G -->|skip plugins| H["Run regression"]
G -->|build plugins| H
Loading
High-Level Assessment
The following are alternative approaches to this PR:
1. Explicit local-image option
➕ Distinguishes local references from published tags without relying on colon detection.
➕ Avoids classifying digest-pinned published references as local images.
➖ Adds a CLI option and a separate usage path for branch runs.
Recommendation: The current tag-compatible approach is compact, but its colon heuristic treats published digest references as local. If digest references are supported, prefer an explicit local-image option or extend reference parsing before claiming all published references retain their channel; otherwise, document or reject digest references.
Files changed (3) +75 / -66
Enhancement (1) +72 / -63
regression-test.shSelect local or published images for regression runs+72/-63
Select local or published images for regression runs
• Normalizes published tag references, runs colon-bearing local references without a pull, and records the selected image and channel in buildinfo.json. Rejects local images with the default version label, skips their plugin builds, and passes one ImageRef through image inspection and container launches instead of redundant tag arguments.
README.mdDocument the local-image build and run procedure+2/-2
Document the local-image build and run procedure
• Explains commit-based version labels, local-channel isolation, the default-label refusal, and why local runs skip plugins. Updates the description of how result directory names resolve to commits.
RunRegressionTests rejects only the default local version label, then uses any other label as a
directory name for root-run filesystem operations. If a branch image carries a label such as
../../../../, the results path resolves to / and the later recursive chown changes ownership
across the host.
+ if [[ "$Channel" == "local" && "$Version" == "1.0.0.0" ]]; then+ echo "Local image $ImageRef carries the default version label, rebuild it with --build-arg LABEL_VERSION" >&2+ exit 1+ fi
Evidence
The new local-image path accepts a non-default label from an image selected by the caller. The
Dockerfile permits that label to be supplied at build time; the harness runs as root and
concatenates it into a path used by recursive chown.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
A local image's version label can escape the results directory and reach root-run filesystem operations.
## Fix Focus Areas
- RegressionTests/regression-test.sh[337-350]
- RegressionTests/regression-test.sh[382-384]
## Recommended Fix
Before constructing VersionDir, reject any version that is not a safe single directory component. Apply the check to every image, including local images, and fail before creating directories or changing ownership.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
2. Pinned published images skip their pull✓ Resolved
Description
The tag normalization removes the published repository only when it is followed by :, so a
published repository@sha256: reference reaches the colon-based local-image branch. Such a run
skips the pull and plugin builds, and if the image is already present, records local as its
channel rather than identifying it as published.
A digest reference retains its sha256: colon after the two prefix removals, making the local
branch inevitable. That channel controls pulling, plugin builds, and the metadata filter used to
select runs.
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution
## Issue description
Digest-pinned references to the published repository are classified as local images, bypassing the pull and changing run metadata.
## Fix Focus Areas
- RegressionTests/regression-test.sh[403-413]
- RegressionTests/regression-test.sh[331-334]
## Recommended Fix
Parse references to the configured published repository before applying the local-image colon rule. Preserve digest references as the image to pull and run, and assign a published channel rather than `local`; cover both registry-qualified and unqualified forms.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Tip of the day
💡 Did you know, you can tweak Display settings with a live preview to see your comment before it ships
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
A moderate active finding shows digest-pinned published repository references are misclassified as local images, causing the regression harness to skip required pull and plugin-build checks.
This pull request enables the regression harness to run locally built branch images before publication while preserving published-image behavior and result tracking.
Changes:
Supports local image references without pulling them.
Records local-run metadata, validates version labels, and skips plugin builds for local images.
Refactors internal functions to use a complete image reference.
Updates the regression README and operations runbook.
Outstanding issues remain around digest-pinned references, the valid published local tag, and tag-only README wording.
File
Review summary
RegressionTests/regression-test.sh
Adds local-image handling and image-reference normalization. Two moderate P1 issues remain: digest-pinned published references can be misclassified as local, and the published local tag conflicts with the local-image sentinel.
RegressionTests/README.md
Documents local image usage, labels, channels, and result directories. A P2 documentation issue remains because some sections still describe inputs only as Docker tags.
OPERATIONS.md
Updates the regression-testing runbook for published and locally built images.
A recorded local strict-review pass covers head e63ca3b47bcba8cccf08c02ca1284584411fde00, the content this pull request carries at that commit against develop, and it recorded 1 finding.
A recorded local strict-review pass covers head 259f734ad8c1c7c893bd7c0906b159a65803ff53, the content this pull request carries at that commit against develop, and it recorded 0 findings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
commentsPermits the comment lines the pull request adds or edits, which the prose gate otherwise refuses
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ptr727
added
the
comments
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
The regression harness always ran
docker pull docker.io/ptr727/plexcleaner:<tag>. So a branch could only get a regression run after it merged and a develop build published it. #1192 and #1193 each need a run before they merge, so this lets the harness run a locally built branch image.tagholding a colon names a local image reference, such asplexcleaner:local, because no Docker tag can contain a colon. It runs without a pull, records"Channel": "local"inbuildinfo.json, and skips the plugin builds. A plugin builds from the release tag its image version names, and a branch build has no release tag.find_runmatchesChannelexactly, so a tool picking the newestdeveloprun never picks a local one.docker.io/ptr727/plexcleaner:Xandptr727/plexcleaner:Xrun as the tagX, so they keep their channel and results directory. A digest reference,[docker.io/]ptr727/plexcleaner@sha256:<digest>, runs as published and pulls by digest.mkdir,cp, andchown -R, so it must be a plain name,^[0-9A-Za-z][0-9A-Za-z._+-]*$. That also keeps a quote out of thebuildinfo.jsonheredoc. A local image's label must also be a commit hash,^[0-9a-f]{7,40}$. That refuses the Dockerfile default1.0.0.0, which every unlabelled build would share, and any published version, which always holds a dot. So a local run never lands in a published build's results directory. The README has you commit first, then build with--build-arg LABEL_VERSION=$(git rev-parse --short=10 HEAD). The results directory is then the commit, andgit rev-parse <version>^{commit}resolves it the same way it resolves a published run's tag.Tagparameter in favor of oneImageRef, and their signature comments change to match. The PR carries thecommentslabel for that reason.Verification:
shellcheckandshfmt -dare clean, and so areprose_lint.py --diff origin/develop --allow-commentsandmarkdownlint-cli2.--helpand the unknown-argument refusal are unchanged.docker,zfs,git,dotnet,id, and filesystem writes, withHostConfigredirected to a scratch directory. A local image skips the pull and the plugins, and recordsChannel: localand its own reference. Both published reference spellings pulldocker.io/ptr727/plexcleaner:<tag>and record the tag as their channel. A digest reference pulls by digest. A missing local image, a local image labelled with the default or a published version, and a label such as../..each exit 1.Findings from the local review
introduced, fixed in e63ca3b: a digest-pinned published reference ran as local. Copilot and Qodo raised it too.pre-existing, declined: an image the harness runs can plant a symlink in a results directory through the existing read-write--volume "$HostConfig:$ConfigPath:rw"mount. A later rootcpor redirect then writes through it. The mount, thecp, and the redirect all predate this change. Running an image already trusts its code with read-write media and results mounts, so a malicious image is outside the harness's threat model, perGOVERNANCE.md"Trust Boundaries and Hardening Effort".WriteBuildInfoand thedocker runcalls. That would need the maintainer to retag an image on his own host mid-run, which the same section rules out guarding.Local review: 5
agent-skillpasses at thefrontiertier, the last over content digest021d935ac4c5.🤖 Generated with Claude Code