Repository navigation
feat(validation): validate Hub Space sources and gate unverified HF runtime - #1525
Draft
burtenshaw wants to merge 10 commits into
Draft
burtenshaw wants to merge 10 commits into
burtenshaw wants to merge 10 commits into
Conversation
burtenshaw
force-pushed
the
ben/rfc008-hf-sandbox-provider
branch
from
October 28, 2026 17:22
a4784de to
5339417
Compare
burtenshaw
force-pushed
the
ben/rfc008-episode-oracle
branch
from
October 28, 2026 17:22
5503a92 to
70b2484
Compare
burtenshaw
changed the title
burtenshaw
force-pushed
the
ben/rfc008-hf-sandbox-provider
branch
from
October 28, 2026 17:25
5339417 to
95e5db6
Compare
burtenshaw
force-pushed
the
ben/rfc008-episode-oracle
branch
from
October 28, 2026 17:25
70b2484 to
457d3e3
Compare
`openenv validate --space <id> [--revision] [--flavor]` downloads the Space at an exact revision, runs static checks on those files, and runs runtime checks against the Hub-built image in a dedicated huggingface_hub Sandbox on the requested HF Jobs flavor, including GPU flavors. The run refuses a Space that moved past the validated revision, never forwards the host token, and confirms job cancellation through the Jobs API. Checks the Jobs VM cannot evidence (network policy, host containment, resource bounds) report the missing provider capability. Live runs are currently blocked by an HF Jobs Sandbox issue: with a uvicorn server running inside a dedicated sandbox, its port proxy and exec API start returning 404 after a few requests. Part of RFC 008 Level Two (#778). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ting `--space --revision main` passed the branch name to the provider, which compared it with the Space's commit hash and refused the run as "Space moved". The CLI now resolves the revision to its commit hash once, before downloading, and uses that hash for the download, the provider and the report target. The provider rejects anything but a full commit hash. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`snapshot_download(local_dir=...)` keeps timestamped download metadata under `.cache/huggingface` in the folder, so two downloads of the same Space commit produced different source digests. The CLI now removes that directory, which huggingface_hub reserves and never commits, before validating the tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The provider compared the Space's commit once during build, then started the mutable `hf.co/spaces/<id>` image and reported the validated commit as if the image were pinned. HF Jobs take a Space image by name only and report no digest, so the image cannot be pinned. The provider now checks immediately before and after the job starts that the Space is at the validated commit and running a Hub build of it (stage RUNNING, built from that commit), stops the job and reports runtime.startup ERROR otherwise, and records the image as unverified. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Join Dockerfile continuations and use the final stage command, preserving inheritance from earlier stages and refusing unsupported entrypoints. Keep only the first 64 KiB of server output while draining the rest so logging cannot fill the sandbox disk or break the server pipe.
Matching Space source and runtime revisions do not bind a Job to an immutable image. Report a named immutable_image SKIP before creating any job, keep launch specifications digest-only, and stop reporting the requested source commit as observed job build provenance. Copy cached source snapshots into an isolated validation tree so SDK metadata stays out of source digests without deleting files tracked by the repository.
Accept a durable report target separately from the validated package path. Construct returned and bundled reports with the same identity while preserving exact source hashing.
Pass the resolved Space source identity into validation before reports are serialized. Cover static and runtime refusal reports, returned objects, printed JSON, source hashes and bundle checksums.
Stop command parsing at a heredoc marker before payload lines can become fictitious stages. Report a named capability SKIP and preserve the supported final-stage parser behavior.
Classify absent runtime stage or source revision as the named unsupported space_runtime_metadata capability. Preserve errors for explicit contradictions and retain the immutable-image refusal without launching jobs.
burtenshaw
force-pushed
the
ben/rfc008-hf-sandbox-provider
branch
from
October 28, 2026 18:34
95e5db6 to
986af41
Compare
This was referenced Oct 28, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
--space,--revisionand--flavorfor Hub Space source validation, followinghf downloadandhf jobs runconventions. Resolve revision names once to a full commit and hash a standalone repository snapshot, preserving tracked files while excluding SDK bookkeeping.Draft: HF runtime is disabled until a trusted source-commit-to-immutable-image binding and direct digest launch are available and qualified. Matching Hub repository/runtime SHAs does not establish job image provenance. Shared launch specifications remain digest-only.
Evidence
Regressions cover named revisions, repeat-download hashes, tracked SDK-metadata paths, multiline/final-stage Docker commands, unsupported command settings, and a real local 2 MiB output producer whose retained log stays at 64 KiB. Returned, printed, saved and bundled reports retain the same resolved Space identity. Unsupported heredocs and unavailable runtime metadata produce named SKIPs; explicit revision/stage contradictions remain errors. Runtime refusal is tested before job creation.
Test Plan
986af419, artifact checksums verified.Reference Linux/amd64 runtime CI passed at
986af419: 806 fast, 35 protocol and all 44 required Docker cases, with complete artifact verification: https://github.com/openenv/OpenEnv/actions/runs/40746462903.These local and Linux CI suites do not qualify live HF execution.
RFC Status
RFC 008 / #778 remains In Review. Live HF CPU/GPU execution, transport and cleanup remain unqualified. The earlier uvicorn/proxy 404 issue was not retested. Network/host/resource enforcement remains unsupported by this provider.
Merge Danger
Door: two-way. Blast Radius: Space source validation and draft HF provider.