Repository navigation
Automated Sync from main to stable - #40
Open
rhods-devops-app[bot] wants to merge 584 commits into
Open
rhods-devops-app[bot] wants to merge 584 commits into
rhods-devops-app[bot] wants to merge 584 commits into
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Important Review skippedBot user detected. To trigger a single review, invoke the
 |
rhods-devops-app
Bot
added
gap-ca30f490af954107a61b9ba559610f6f
gap-e187ca30fe3b486f852ba682e008f47c
gap-0255d890cb1644dd81c2c60126c92894
gap-120b108c4b714a578079303b54baf0c3
gap-2fef5f03b7b34b6ba04033965b1c7c72
gap-09c808e55d77403cbbc316f7b737037c
gap-0143b9f7f21c4a9da86546d97eaf77f1
gap-65c20b4a54fb4221ae161d227cd3402c
labels
…DIA#4947) In a git worktree .git is a file, so the hard-coded ../../.git/* rerun-if-changed paths never exist and cargo rebuilds openshell-core and every dependent crate on each invocation. Resolve HEAD and logs/HEAD from the git dir and tag refs, packed-refs, and the checked-out branch ref from the common git dir, and only emit paths that exist. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
…VIDIA#4744) * refactor(gator): separate runtime assignments from tooling images (WIP) Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * feat(agents): support Kubernetes image delivery and reusable tooling Signed-off-by: John Myers <johntmyers@users.noreply.github.com> * fix(agents): use pushed Podman manifest digest Signed-off-by: John Myers <johntmyers@users.noreply.github.com> --------- Signed-off-by: John Myers <johntmyers@users.noreply.github.com> Co-authored-by: John Myers <johntmyers@users.noreply.github.com>
NVIDIA#5065) * feat(cli): support fixed callback port for browser login in containers Browser login (OIDC Authorization Code + PKCE and the edge/mTLS browser flow) binds its callback listener to an ephemeral port on 127.0.0.1. When the CLI runs inside a container, the wrapper cannot know a random port in advance, so it cannot publish it to the host and the browser never reaches the callback. Add a shared bind helper honoring two environment variables: - OPENSHELL_CALLBACK_PORT pins the listener to a fixed port (validated integer 1..=65535) so a wrapper can publish -p 127.0.0.1:<port>:<port> before login starts. - OPENSHELL_CALLBACK_BIND_ADDRESS overrides the bind address (validated IP address) so user-mode container networking (pasta/slirp4netns), which can only reach 0.0.0.0 listeners, can forward to it. Defaults are unchanged: with both variables unset the listener binds 127.0.0.1:0 exactly as before, and the bind address never widens to 0.0.0.0 without the explicit variable, since a wildcard bind on a host machine would expose the callback server to the local network. The browser-facing redirect URL stays http://127.0.0.1:<port>/callback even for a wildcard bind, so no identity-provider allowlist change is needed. A busy fixed port or an invalid value fails with a clear error instead of falling back silently. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * fix(auth): support IPv6 callback targets in browser login flows An IPv6-only callback bind (OPENSHELL_CALLBACK_BIND_ADDRESS=::1) was accepted but both browser flows still directed callbacks at 127.0.0.1, so the listener was unreachable and login timed out. Route callbacks to the address the listener actually bound: - The CLI derives the browser-facing callback target from the bound address. Wildcard binds map to the matching loopback (0.0.0.0 to 127.0.0.1, :: to [::1]) because a wildcard listener accepts loopback connections of its family; IPv6 addresses render bracketed per RFC 3986/RFC 8252. - The OIDC redirect URI therefore becomes http://[::1]:<port>/callback for ::1-bound listeners, and the bundled Keycloak realm now allows http://[::1]:* redirects for the CLI client alongside the existing IPv4 entries. - The edge flow passes the unbracketed loopback literal to the gateway connect page as callback_host. The page strictly validates it to a loopback IP literal (anything missing, non-loopback, or malformed falls back to 127.0.0.1) before embedding it in the fetch URL and CSP connect-src, so a crafted parameter can never direct the embedded edge token to an external host. The host is preserved across the waiting-page refresh, and older gateways that ignore the new parameter keep the historical 127.0.0.1 behavior. Defaults are unchanged: the listener still binds 127.0.0.1 on an ephemeral port unless the environment variables say otherwise. Adds a callback-reachability test (connecting to the advertised [::1] target succeeds while 127.0.0.1 is refused for a ::1 bind), host-derivation table tests, OIDC redirect tests for ::1 and :: binds, edge URL tests, and server-side validation and render tests. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> * docs(auth): scope IPv6 support around Chrome's CSP limitation Review feedback on b676200: Chrome does not support IPv6 literals in CSP host-source expressions, so the edge flow's connect page blocks the callback POST to http://[::1]:<port> with a connect-src violation even though the policy allows it. The CLI listener is reachable; the browser refuses to fetch. Keep the callback routing as-is (correct per the CSP specification, and the failure mode is an explicit page error rather than a hang) and scope the claim precisely instead: - The connect page CSP always keeps the IPv4 loopback source so a dropped IPv6 source expression cannot degrade the policy to empty. - The auth reference and PR description now state that IPv6 callback binds are fully supported for OIDC login but are best-effort for edge login until browsers implement IPv6 CSP source expressions; all IPv4 paths, including the container use case, are unaffected. Signed-off-by: Jesse Jaggars <jjaggars@redhat.com> --------- Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
…ing with Keycloak (NVIDIA#4891) * feat(testkit): pass extra environment variables to the CLI Add Cli::run_with_env and Context::cli_with_env so tests can hand credentials to the CLI by name instead of on its command line. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * feat(tmachine): add an Ubuntu Docker environment that runs Keycloak Environment setup can now take inputs, like installers do, and hashes them only when present so existing cached disks stay valid. ubuntu-docker-rootful-keycloak runs the Docker setup plus a Keycloak playbook that starts the container with a restart policy, since setup is cached as a disk, and writes how to reach it as OPENSHELL_TEST_* lines in /etc/openshell-test/env.d/keycloak.env. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * feat(test-runner): pass the variables an environment publishes to tests After boot, read every /etc/openshell-test/env.d/*.env file, accept only OPENSHELL_TEST_* names, and add the variables to the testkit-ready and nextest commands through guest_env. Errors name the key, never the value. The runner doesn't know which services an environment runs; suites read the variables, so they also run against a deployment that sets them. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(features): port Keycloak provider refresh to testkit Add a keycloak features crate with a suite-local KeycloakContext fixture that reads OPENSHELL_TEST_KEYCLOAK_*, waits for the realm to answer, and provides password grants, revocation and the token endpoint. Port the provider refresh test onto it. Align the features lockfile with the root and add it to the alignment check. The old provider-refresh crate stays until the required lanes switch. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(tests): run the Keycloak features suite in E2E Suites Add a matrix row for features/keycloak on ubuntu-docker-rootful-keycloak, include the Keycloak realm in the tmachine disk cache key, and document environments that provide services in TESTING.md. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(testing): describe how to add a feature suite Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(tmachine): publish test variables through a shared task file Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(test-runner): keep published test variables out of SSH commands The runner passed the variables in env.d on the nextest command line, so an SSH timeout or launch failure printed their values, which can be credentials, in the error context. The runner now only checks the files; the guest shell exports them before starting nextest, so only the gateway name and CLI path appear in the command. Lines are read literally on both sides: no trimming, and only empty lines and lines starting with # are skipped. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(features): fit Keycloak readiness in the per-test timeout The fixture waited up to three minutes for Keycloak inside the test, but the runner's nextest profile kills a test after two. Wait up to 90 seconds so a slow Keycloak fails with a readiness error instead. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(test-runner): run the export script test with shell builtins only The Nix build runs the runner's tests in a sandbox without env on PATH, so the test failed there and the runner didn't build. Print the exported variables with printf instead. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(features): name the Keycloak suite after what it tests The suite was named after the service it needs rather than what it tests. Rename it to keycloak-provider-refresh and move the Keycloak fixture into a support crate at tests/support/keycloak, so other suites that need an OIDC issuer can depend on it. TESTING.md now asks for a descriptive suite name instead of prescribing one. Lint and format the new crates in CI and mise. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(features): bound each Keycloak request in the fixture poll_until checks its deadline only between attempts, and curl had no time limit, so a Keycloak that accepts connections but stalls could hold one attempt until nextest killed the test. Give each request 10 seconds. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(tests): key the tmachine disk cache by environment and installer Both E2E suites rows computed the same cache key. Only the first job to save it got its disks in, and actions/cache never saves after an exact hit, so the other row rebuilt its setup and install layers on every run. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(testkit): keep the gateway selection over caller CLI env run_with_env applied the caller's variables last, so one named like the gateway, endpoint or workspace variable could silently point the CLI at another gateway. Apply them first so testkit's selection wins. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * fix(features): leave the test body room after Keycloak readiness poll_until checks its deadline only between attempts, so a 90 second wait could run for about 102 seconds with the request time limit and the poll interval, leaving the test body under 20 seconds of the 2 minute budget. Wait 75 seconds and spell out the margin. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * ci(tests): include config.nix in the tmachine disk cache key tmachine's setup hash covers each environment's playbook list and setup inputs, which config.nix declares. Changing only config.nix left the cache key unchanged, so the rebuilt layers were never saved again. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(testing): add the lint step for new feature suites CI formats and lints feature suites one package at a time, so a suite added by following TESTING.md got no fmt or clippy coverage. Also say what test_env actually guarantees about published values. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(features): report Keycloak suite cleanup failures The provider and profile cleanup discarded both errors and failed exit codes, so leftovers went unnoticed. Print them without failing the test. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
Signed-off-by: red-hat-konflux <126015336+red-hat-konflux[bot]@users.noreply.github.com>
Signed-off-by: konflux-internal-p02 <170854209+konflux-internal-p02[bot]@users.noreply.github.com> Co-authored-by: konflux-internal-p02[bot] <170854209+konflux-internal-p02[bot]@users.noreply.github.com>
NVIDIA#5089) Keycloak 24 rejects http://[::1]:* (and every other port wildcard on an IPv6 literal) when importing the realm, so it fails to start with "Invalid client openshell-cli: A redirect URI is not a valid URI". That broke every lane that loads scripts/keycloak-realm.json, including the features/keycloak suite and the OIDC Docker E2E jobs. Restore the IPv4 loopback and localhost entries only, and correct the authentication docs: IPv6 callback binds against the bundled realm need an exact http://[::1]:<port>/callback redirect URI. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
… docker tag to v9.9-1793849542 (#126) Signed-off-by: rhods-devops-app[bot] <131765678+rhods-devops-app[bot]@users.noreply.github.com> Co-authored-by: rhods-devops-app[bot] <131765678+rhods-devops-app[bot]@users.noreply.github.com>
…g to v9.9 (#128) Signed-off-by: rhods-devops-app[bot] <131765678+rhods-devops-app[bot]@users.noreply.github.com> Co-authored-by: rhods-devops-app[bot] <131765678+rhods-devops-app[bot]@users.noreply.github.com>
… (NVIDIA#5498) Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
Update hickory-proto from 0.26.1 to 0.26.2 in the root, governance example, and five aligned test workspace lockfiles. This resolves RUSTSEC-2026-0487, RUSTSEC-2026-0488, RUSTSEC-2026-0489, RUSTSEC-2026-0490, and RUSTSEC-2026-0491 in every affected graph. Preserve content-guard's patched 0.26.3. Native Cargo resolution also rebinds existing Windows-only references within their parent constraints; no other package version changes. RSA and SQLite exceptions were reviewed and remain blocked by published parent constraints, with no policy edits. Signed-off-by: Adrien Langou <alangou@nvidia.com>
…IA#5503) * chore(review): prepare label taxonomy and nested SDK topics Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * refactor(review): keep only permanent migration prerequisites Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * docs(review): replace broad SDK topic with language topics Signed-off-by: Jim Meyer <jimeyer@nvidia.com> --------- Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
* feat(labels): add bounded GitHub transport Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): bound transport cleanup and validate continuation links Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * feat(labels): validate classification and plan reconciliation Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): require trusted malformed removal candidates Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * feat(labels): prepare expandable evidence and issue eligibility Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): preserve evidence consistency and bound expanded packets Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * feat(labels): persist trusted examination checkpoints Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): validate source coverage before normalization Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): recognize verified current stale closure protocol Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): allow bounded stale closure request skew Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * feat(labels): reconcile item labels with checkpoint recovery Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): bind assessment material and preserve inspection boundaries Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): bind evidence citations to delivered source facts Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): finalize exposure after core packet hydration Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): return successful exit status for previews Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * feat(skills): extract shared issue and PR labeling Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * docs(skills): clarify assessment-only continuation Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * docs(labels): preserve approved design and implementation contracts Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * docs(labels): clarify HTTP write pacing for mutation batches Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labeler): verify additions before destructive reconciliation Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * refactor(labels): simplify evidence and semantic freshness Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * chore: exclude local superpowers planning documents Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * docs(skills): remove issue labeling paragraph hard wrapping Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): follow accepted-work library rename after rebase Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * refactor(labels): simplify guidance and stale-closure detection Signed-off-by: Jim Meyer <jimeyer@nvidia.com> * fix(labels): pass Python formatting lint and type checks Signed-off-by: Jim Meyer <jimeyer@nvidia.com> --------- Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
Signed-off-by: Jim Meyer <jimeyer@nvidia.com>
…the middleware (NVIDIA#4359) * feat(middleware): add HTTP middleware protocol 2 Add EvaluateHttp, a streaming HTTP middleware protocol with BUFFERED and STREAM body modes, next to the released HTTP protocol 1, which keeps its engines unchanged and is deprecated for removal in 0.2.0. - One EvaluateHttp stream per HTTP message and stage, or per uninspectable connection. A preflight selects continue, inspect, or reject; late header mutations apply before the head commits. - A service implements one HTTP protocol, chosen per binding with http_protocol_version, and a protocol 2 service requires the openshell.supervisor-middleware.http-v2 capability. The gateway delivers each service's protocol to supervisors. - Protocol 2 is always fail-closed. The gateway rejects fail_open on protocol 2 services and policies whose overlapping selectors mix protocols; a mixed chain fails closed at runtime. - Protocol 2 middleware decides about uninspectable traffic (tls: skip, h2c, unsupported tunnels, raw TCP, SQL passthrough) at a preflight, and is exempt from the static tls: skip conflict rule. - Request bodies stream to the upstream unless a later step needs the whole body; responses stream with chunked framing or buffer before commit. - Migrate the content guard example to protocol 2, add a Docker e2e suite, and document the protocol. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): select HTTP protocol 2 by binding operation Replace the http-v2 capability and per-binding protocol fields with new binding operations, and stop delivering the protocol to supervisors. - HTTP_REQUEST_V2 and HTTP_RESPONSE_V2 select HTTP protocol 2. Released gateways and supervisors already reject unknown operation/phase pairs at Describe, so the capability, extension_metadata_with_requirements, and MiddlewareBinding.http_protocol_version are removed. - Drop supported_http_body_modes. The preflight offers every mode the message is eligible for, and the stage picks one or continues or rejects. A protocol 2 binding with a zero payload limit is offered no body modes. - Move the tls: skip conflict rule out of the shared policy validator into the gateway's safety validation, which knows each service's protocol. This removes the delivered SupervisorMiddlewareService protocol field, the process-wide protocol 2 set in openshell-policy, and the supervisor's undescribed-service tracking. An undescribed entry follows on_error, which the gateway requires to be fail_closed for protocol 2 services. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): split EvaluateHttp into request and response RPCs Replace EvaluateHttp with EvaluateHttpRequestV2 and EvaluateHttpResponseV2, one RPC per binding operation, as for HTTP protocol 1 and WebSocket. Both keep the shared HttpEvent and HttpResult messages, so the stage pipeline is unchanged; OpenShell picks the RPC from the preflight subject. Traffic OpenShell cannot inspect stays on the request RPC. Only services that bind HTTP_REQUEST_V2 decide about it, so only they are exempt from the gateway's tls: skip conflict rule; a response-only protocol 2 service keeps the rule and its fail-closed entry denies such traffic at runtime. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): call the new contract v2 HTTP hooks Rename "HTTP protocol 1/2" to v1 and v2 HTTP hooks across code, proto comments, docs, skills, and the example. The docs page moves to extensibility/supervisor-middleware/http-hooks-v2, the protocol2 modules become http_v2, HttpProtocol becomes HttpHookVersion, and the mixed-chain reason becomes middleware_hook_versions_mixed. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): select HTTP session hooks by required capability Rename the streaming HTTP hook RPCs to EvaluateHttpRequestSession and EvaluateHttpResponseSession, and drop the unreleased HTTP_REQUEST_V2 and HTTP_RESPONSE_V2 operations. A service keeps its HTTP_REQUEST and HTTP_RESPONSE bindings and selects HTTP session hooks for all of them by listing openshell.supervisor-middleware.http-session in its extension required_capabilities. Released gateways and supervisors already reject unmet required capabilities at Describe, so they never call such a service through the v1 RPCs. Registration rejects a manifest that supports the capability without requiring it. Mark the v1 HTTP hook RPCs, the HttpResponsePreReturn service, and the types only they use deprecated in the protobuf contract. Rename the docs page, module, and e2e suite to HTTP session hooks, and update RFC 0009 so capability-gated contracts and removing deprecated elements in a minor release match the release stability policy. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): begin body stages on every preflight mutation A body stage captured the head right after its own preflight, so its begin event lacked the preflight mutations of later stages even though the delivered head carried them. Keep the final preflight head on the pipeline and begin every stage on it plus earlier stages' late mutations, as the HttpBegin contract states. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(network): check the policy generation before every response write A STREAM response stage can keep emitting output after it consumed the upstream body, and the session response writer checked the policy generation only when it committed the head. Output and the terminating chunk then reached the client under revoked authorization, before the final check. Check the generation before every write instead, and treat the response as committed only once a byte was written. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(network): complete session hook messages only after every stage With a declared output length, a STREAM stage could emit every byte and then reject at input_end. The relay had already written a complete Content-Length message, so the sandbox saw a successful response, and on uploads the upstream received a complete request it could act on or answer. Hold back what would complete the message until every stage returns its terminal result: the final byte of a declared length, or the whole head of a declared empty body. The chunked terminator already waited. An empty response body that is rejected still gets the canonical 403. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(examples): split expanded content guard output within the chunk limit Redacting a term shorter than the marker makes STREAM output longer than its input, so one output chunk could exceed the max_chunk_bytes the preflight offered and fail the message closed. Split every output chunk within that limit. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): queue HTTP session hooks and explain failures to the sandbox Address load-test findings on HTTP session hooks: - Give HTTP session hooks their own admission budgets: 64 slots for preflight and BUFFERED bodies, and 256 for STREAM bodies so long-lived server-sent events cannot starve short messages. A message waits for a slot within the chain deadline instead of failing at once, releases it when no stage inspects the body, and frees its BUFFERED slot when the last BUFFERED stage ends. Uninspectable decisions use the same budget. - After an early middleware response (preflight rejection or failure, 503, mid-stream failure, or 408), shut down writing and discard what the sandbox still sends, within time and byte bounds, so the sandbox reads the response instead of a connection reset. - Fill originating_process (binary and ancestors) in request and response preflights, as the uninspectable preflight already did. - Return a platform-owned failure_code from a closed set, and the failed stage's policy entry name, in 403, 502, and 503 failure bodies. - Document raising gRPC message limits for 4 MiB BUFFERED bodies, and raise them in the content guard example. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): give each HTTP session hook RPC its own messages Address the proto review of the HTTP session hooks contract: - Split the shared HttpEvent/HttpResult into per-RPC messages (HttpRequestSessionEvent/Result and HttpResponseSessionEvent/Result) with direction-specific preflights, so a request stream cannot carry a response subject and the reverse. Body events and results stay shared. - Add phase to both preflights, as the WebSocket preflight has. - Add an explicit cannot_inspect result, valid wherever reject is, and drop the FAILED_PRECONDITION convention; every gRPC status now maps to middleware_unavailable. - Offer no body mode for a request whose Content-Encoding is not identity (ENCODED), as for responses: middleware would only see the encoded bytes. - Send declared_body_bytes, and apply OVER_LIMIT, only to stages before the first one that inspects the body, which may replace it. - Remove the queue limits from HttpBodyLimits; they had no defined meaning for a service. - Clarify comments: BUFFERED fails over its limit instead of falling back, finish requires output_start, and unavailable_body_modes is empty for a zero payload limit. - Document that policies stored before a service changed hook version must be re-applied. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(network): include the originating process in response preflights The main L7 response path built its middleware request context by hand with originating_process set to None, so HTTP session hook response preflights lacked the process that request preflights already carry. Build it with middleware_request_context, as the other paths do. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): retain response delivery blocking and consolidate docs Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): release the preflight slot while waiting for a stream slot A message whose stages selected STREAM kept its preflight slot while it waited, up to the chain deadline, for one of the stream slots. With every stream slot taken by open server-sent-event responses, waiting messages could hold all preflight slots and starve BUFFERED messages and uninspectable-connection decisions. Take a free stream slot immediately when one is available. Otherwise release the preflight slot before waiting, and take one again afterwards only if a BUFFERED stage runs. No message holds a preflight slot while it waits for a stream slot, so the two waits cannot deadlock. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): protect request Content-Encoding from session hook stages Response preflight already treats Content-Encoding as read-only. On requests a stage could still write or remove it. That was not a bypass, because body-mode offers come from the original head, but a stage could label a body it replaced as encoded, or drop the label from an encoded body. Session hook request heads now protect Content-Encoding. Legacy request hooks keep their released rules. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * feat(middleware): record cannot_inspect diagnostics HttpCannotInspect was empty, so a stage could not attach findings or a reason code for operators the way HttpReject can. Add diagnostics to it. OpenShell validates and records them like a rejection's, and the sandbox still sees only failure_code middleware_cannot_inspect. Also update the HttpPreflightResult comment for the per-RPC split: requests deny with reject, responses with block_delivery, and either can return cannot_inspect. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * docs(middleware): cover cannot_inspect for uninspectable traffic and gateway restarts Uninspectable preflights can also return cannot_inspect, which refuses the connection as a failure. The gateway reads each service's Describe manifest only at startup, so re-applying policies after a service changes hook version validated them against the stale manifest. Tell operators to restart the gateway first. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * feat(middleware): let session hook stages own integrity headers Response session hooks stripped ETag, Content-MD5, Digest, Content-Digest, Repr-Digest, Signature, and Signature-Input whenever a stage replaced or streamed the body, and stages could only remove them. A STREAM stage that passed the body through unchanged still lost the upstream's validators, and no stage could supply correct new values. Requests never stripped them. Session hooks now leave these headers alone in both directions, as Envoy's external processor does. A stage that changes the body removes them or writes new values in buffered_result or output_start; a STREAM stage can read all input before output_start to compute one. Legacy response hooks keep their released stripping and remove-only rules. The content guard example removes them when it replaces a BUFFERED body, and at output_start when it streams in redact mode. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * docs(middleware): list cannot_inspect where proto comments name stage choices The UninspectableTraffic comment and both preflights' permitted_body_modes comments still said a stage that finds no suitable mode continues or denies. It can also return cannot_inspect. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * docs(middleware): recreate sandboxes after a service changes hook version Supervisors also cache each service's Describe manifest and rebuild their registry only when the set of service registrations changes. Restarting the gateway and re-applying policies leaves running sandboxes on the old manifest. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * test(e2e): find the middleware host when Docker discovers the gateway Since NVIDIA#4016, host e2e runs omit the Docker driver's grpc_endpoint and let the driver discover the gateway. The HTTP session hook test required the key and panicked. Fall back to loopback, which the host-networked supervisor shares with the gateway, and read the key only from the Docker driver table. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): deny uninspectable messages with reject The cannot_inspect result replaced a FAILED_PRECONDITION status convention after review, which asked only for an explicit, typed signal whose effect is the reason. A separate result went further: it added a third terminal result to every phase and a failure response (502 on responses, platform-owned failure_code) for what is a decision the stage makes. Remove HttpCannotInspect and the cannot_inspect result field from both session result messages. They are unreleased, so they are deleted, not reserved. A stage that cannot inspect a message rejects it, or blocks delivery, with a reason_code that says so. The content guard example uses reason code cannot_inspect. The platform failure_code middleware_cannot_inspect remains for messages OpenShell cannot show to a stage. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): split HTTP session RPCs into dedicated service Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): decide about uninspectable traffic in a dedicated hook Uninspectable connections (tls: skip, h2c, unsupported tunnels, raw TCP, SQL passthrough) are not HTTP, so they no longer use an HTTP session preflight. A service binds UNINSPECTABLE_NETWORK/PRE_FORWARD and serves the unary UninspectableNetworkMiddleware.Evaluate on the same endpoint. The hook carries only connection metadata, can be a service's only binding, and always fails closed. Entries without the binding keep their on_error behavior: fail_closed denies, and fail_open relays with a bypass finding. That fail_open exception is deprecated for removal in 0.2.0. The gateway tls: skip conflict rule now exempts middleware that binds the new hook. The tls: skip decision runs before 200 Connection Established, so a denial is a plain 403. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): align HTTP denial naming Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): defer inspection decisions until stage input is ready Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * docs(middleware): document the UTF-8 rule for HttpHeader values The L7 parser rejects request heads that are not valid UTF-8 before any middleware runs. A response head that is not valid UTF-8 fails an HTTP session hook chain closed with middleware_cannot_inspect, while v1 response hooks follow on_error. Session hooks also fail a message with trailers that are not valid UTF-8. State this on HttpHeader so service authors know the string type matches what OpenShell forwards. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): tidy the unreleased HTTP session hook contract HttpRequestSessionPreflight skipped field 2 after the uninspectable subject moved to its own hook. These fields were never released, so renumber them to match HttpResponseSessionPreflight instead of leaving a gap. Rename UninspectableTrafficReason to UninspectableNetworkReason to match UninspectableNetworkMiddleware and the other new UninspectableNetwork names. Document behavior the contract left implicit: a trailer write cannot add a name, Content-Length framing drops trailers, and Process.pid is 0 when unknown. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): report the withheld STREAM output cap per route Routes that hold request output until the chain finishes capped it at 4 MiB, while the contract said STREAM has no total size limit. A pass-through STREAM stage on a SigV4 route therefore failed requests between 4 and 10 MiB that the route accepts without middleware, and the stage had no way to know. Withhold output up to the route's own buffering limit: 10 MiB for SigV4 body signing, 256 KiB for body credential rewriting, and 4 MiB otherwise. Request preflights report that limit to STREAM stages as the new HttpBodyLimits.max_output_body_bytes, so a stage can decide at preflight. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * docs(middleware): attach trailer mutation comments to their fields The trailer mutation rules added in 9f7a04a landed on HttpInputChunk.data and inside the HttpDenial comment. Move them onto HttpBufferedResult.trailer_mutations and HttpFinish.trailer_mutations, and restore the HttpDenial comment. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): ask for identity-coded responses when a session stage can inspect bodies HTTP session hooks offer no body mode for a response with a content coding other than identity. The sandbox controls Accept-Encoding, and common clients (requests, httpx, fetch) send gzip by default, so most compressed responses reached response stages uninspectable: a guard either blocked them with cannot_inspect or let them through unread. When the response chain has an HTTP session hook stage that may select a body mode, the relay now replaces every Accept-Encoding field with identity. It does so as the last header step on both the buffered and the streamed request path, after request-stage mutations and before credential injection, so a request stage cannot undo it. Request middleware still sees the sandbox's value. v1 hooks and bindings with max_payload_bytes 0 are unchanged. An encoded response now means the upstream ignored identity. The e2e upstream gains a /compressible route that gzips when asked, and the workload asks for gzip as common clients do. Without the change the guard answers 403 cannot_inspect; with it, the response is redacted. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * feat(middleware): show response stages the forwarded request headers A response stage that needs the request's headers, such as a guard that judges a response against what was asked, had to keep per-request state keyed by request_id and route both streams to the same replica. HttpResponsePreflightHead.request_headers now carries the request headers as the request middleware chain forwarded them: after every request stage's mutations, including a streamed body's late mutations, and before OpenShell injects credentials or replaces Accept-Encoding. They have the omissions and limits of the request preflight's headers, and a value that contains a credential placeholder reads [REDACTED], as the response target's query already does. The relay hands response middleware a request that may already carry token grants, under header names the request-header sanitizer does not know, so the head is recorded before injection: by HttpMiddlewareExchange when request middleware allows the request, and before prepare_inspected_request on the inspected relay path. A live body applies its late mutations to that record when its head commits, before the upstream can answer. A response chain with no recorded head fails closed with middleware_cannot_inspect, as do request headers over the head limits. The response preflight's context comment now says request_id matches the request's preflights. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): hide credential placeholder headers from session hooks HTTP session hook heads already omit Authorization, Cookie, and other credential headers, but a credential placeholder in a custom header, such as X-Api-Key: openshell:resolve:env:API_KEY, reached request stages as-is and response stages as [REDACTED]. Whether a stage saw a placeholder depended on which header carried it. A request header whose value carries a credential placeholder, including one inside a Basic credential, is now a credential header for HTTP session hooks. The request preflight, begin, and the response preflight's request_headers omit every value of that name, and the pipeline rejects a stage mutation that writes or removes it with header_mutation_protected_header, so a stage cannot replace or drop a credential reference it cannot see. The [REDACTED] sentinel is gone. Like Envoy's ext_proc forwarding rules, OpenShell drops headers rather than masking their values; a typed list of withheld names can be added later if a stage needs to know they were present. v1 HTTP hooks keep their released behavior and still receive these headers. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): remove the uninspectable network hook The hook let middleware allow connections OpenShell cannot inspect (tls: skip, h2c, unsupported tunnels, raw TCP, SQL passthrough) by metadata alone. No published middleware uses it, and the mixed-port cases it served are better handled by selector changes, so remove it before it ships. Uninspectable traffic goes back to the chain-level on_error rule: a fail_closed entry denies the connection, and an all-fail_open chain relays it with a bypass finding. That fail_open exception is no longer deprecated. The tls: skip overlap rule returns to shared policy validation, so supervisors enforce it again alongside the gateway. The tls: skip gate still runs before 200 Connection Established, so a denial is a plain 403. The proto service, operation, phase, and enums are deleted rather than reserved because they were never released. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(policy): reject ports in middleware selector patterns Middleware selectors match hostnames on every port, but a pattern such as api.example.com:443 parsed as a hostname that never matches, so the entry was silently never selected. Reject ':' in include and exclude patterns at gateway and supervisor validation instead. IPv6 literals are still accepted. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(policy): keep IPv6 globs in middleware selector patterns Rejecting every ':' outside a complete IPv6 literal also rejected IPv6 globs such as 2001:4860:*, which selectors have matched since they shipped and network endpoints still accept. Treat a pattern containing ':' as IPv6 text instead, and reject only the shapes of a host:port authority: bracketed literals, characters IPv6 text cannot contain, dots before the last ':', a single ':' followed by digits, and invalid literals without wildcards. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * feat(middleware): send the originating process to every middleware hook v1 HTTP request hooks and WebSocket preflights now receive RequestContext.originating_process like v1 response hooks and HTTP session hooks, so every hook sees the same caller. A deep ancestor chain is trimmed, farthest ancestor first, to keep the context within its 4 KiB limit instead of failing the evaluation. Closes NVIDIA#4269 Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(policy): accept only complete IPv6 addresses in middleware selectors IPv6 globs such as cafe:* cannot be told apart from a mistyped host:port, so the heuristic that accepted them let some port patterns through as IPv6 text that never matches. Middleware selectors now accept ':' only in a complete IPv6 address. Docs add the upgrade note for policies with a port or an IPv6 wildcard. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(network): answer a JSON-RPC or GraphQL request timeout with 408 These paths read the whole body before middleware runs, so a request timeout should not occur. If it does, send the same 408 request_timeout as the REST path instead of failing the relay with a generic error. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * docs(middleware): map session hook failures to the sandbox failure_code List request_timeout among the sandbox error codes, and name both the OCSF reason and the failure_code a sandbox sees in the troubleshooting skill. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): rename HTTP body modes to WHOLE_BODY and DUPLEX BUFFERED becomes WHOLE_BODY and STREAM becomes DUPLEX, before the HTTP session hooks ship. WHOLE_BODY matches the v1 WHOLE_BODY_BYTES name, and DUPLEX leaves room for a later PER_CHUNK mode with one result per input chunk, like Envoy ext_proc STREAMED, without two modes both called streaming. Renames the proto enum values, HttpWholeBodyMode, HttpDuplexMode, HttpWholeBody, HttpWholeBodyResult, the whole_body and whole_body_result fields, and max_whole_body_bytes, along with the Rust names, the content-guard body_mode values, tests, and docs. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): rename http-session capability to http-hooks The capability that selects the streaming HTTP RPCs is now openshell.supervisor-middleware.http-hooks. "Session" read as a cookie-style HTTP session and clashed with the WebSocket session RPC. Docs, comments, and errors call the new contract "HTTP hooks" and the deprecated contract "legacy HTTP hooks" instead of "v1". HttpHookVersion and ChainHttpHookVersion rename V1 to Legacy, with the audit label legacy_http_hook; the new contract's label is http_hook. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): rename HttpSessionMiddleware to HttpHooks Finish the http-hooks rename so no HTTP hook name still says "session": - Service HttpSessionMiddleware is now HttpHooks; Http{Request,Response} Session{Event,Result,Preflight} messages are now Http*Hook*. - http_session modules, the e2e test, and the mise task and CI suite are now http_hooks / middleware-http-hooks. - HttpHookVersion and ChainHttpHookVersion rename Session to Current; HeaderAuthority renames Session{Request,Response} to Hook*. - HttpRequestSession and HttpResponsePipelineSession are now HttpRequestPipeline and HttpResponsePipeline, carried in a `pipeline` field; MAX_CONCURRENT_HTTP_MIDDLEWARE_SESSIONS is now ..._MESSAGES. Shared stream names (MiddlewareSessionEnd, session_capacity_exhausted) and WebSocket session names are unchanged. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): name HTTP hook budgets after their slots "Messages" and "streams" read as gRPC messages and streams. The two budgets are buffering slots (preflight and WHOLE_BODY) and streaming slots (DUPLEX or deferred stages), as the docs already call them: - MAX_CONCURRENT_HTTP_MIDDLEWARE_MESSAGES -> MAX_HTTP_HOOK_BUFFERING_SLOTS - MAX_CONCURRENT_HTTP_MIDDLEWARE_STREAMS -> MAX_HTTP_HOOK_STREAMING_SLOTS - http_{hooks,stream}_admission -> http_{buffering,streaming}_admission, with matching reserve helpers and debug fields. The streaming budget comment now says deferred stages also take a slot, matching may_stream(). The docs limits row counts HTTP requests and responses instead of messages. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * docs(middleware): align HTTP hooks contract wording with review feedback State how zero-limit HTTP hooks bindings interact with the operator payload limit, align the HTTP/1.0 DUPLEX rule with the proto, document the 403 for a DUPLEX request denied after upstream output starts, and make failure_code wording consistent across the docs. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * test(middleware): cover zero-limit HTTP hooks bindings and late DUPLEX request denial Pin that a zero-limit HTTP hooks binding is excluded from the operator payload limit comparison, and that a DUPLEX request denied after its output started upstream still gets the canonical 403 and an unterminated upstream body. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * refactor(middleware): give HTTP hook RPCs package-unique names Rename HttpHooks.EvaluateRequest and EvaluateResponse to EvaluateHttpRequestHook and EvaluateHttpResponseHook, and document in supervisor_middleware.proto that RPC names must be unique across the package's services, with the naming pattern for future hook services. Python and Go servers name handler methods after the RPC alone, so one object implementing several services needs distinct names. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(middleware): mark HTTP hooks response denials in OCSF findings Set ActionId::Denied on the HTTP hooks http_response_blocked finding and on the after-commit delivery failure when a stage stopped delivery, so both hook versions emit the same OCSF for the same decision. Assert that a DUPLEX request denied after upstream start leaves the chunked upstream body unterminated. Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> --------- Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
…locking pool (NVIDIA#5217) * fix(sandbox): read exec pipes and PTY masters without blocking pool threads Exec stdin/stdout/stderr and the PTY master were wrapped in tokio::fs::File, so every pending read on an idle pipe parked one tokio blocking-pool thread until data arrived or the peer closed. That pool is shared with the control plane (gRPC exchange, exec spawn/reap, AcceptNetwork), and enough idle exec sessions exhausted it, blocking new control connections. Adopt the descriptors with a readiness-based AsyncFd adapter instead: reads and writes only run when the descriptor is ready, and idle execs park no pool threads. Flush and shutdown keep the tokio::fs::File semantics the consumers were written against, and close-on-drop remains the EOF mechanism, so no consumer changes. An isolated boundary test caps the blocking pool at 16 threads, starts 12 idle execs, and asserts a fresh exec and a control request still complete; without the fix it fails when the sixth idle exec stalls. Signed-off-by: Emilien Macchi <emacchi@redhat.com> * fix(sandbox): defer freeze descendant scans off the supervisor lock transport_disconnected froze the workload while holding the supervisor_connection mutex, and the freeze signaled every descendant through up to four synchronous /proc scans, so losing the supervisor stalled a runtime worker and the lock for as long as the scans took, which grows with the number of processes in the sandbox. Freeze and resume now stop and continue the registered root groups synchronously, so a confirmed recovery still implies the roots were continued, and defer the descendant sweeps to a dedicated thread: the shared blocking pool can be fully occupied, and a sweep queued behind it would leave detached children running while the boundary reports itself frozen. If thread creation itself fails, the sweep runs synchronously rather than leave the freeze unenforced. Each sweep collects the owned set without holding any lock and applies the signal under a signal mutex only while the transition generation it captured is still current, so a stale deferred SIGSTOP can never land after a resume SIGCONT. A sweep that fails to deliver reports the gap as an OCSF process activity naming the recovery. The scans now run under neither supervisor_connection nor the signal mutex; a transition that immediately follows another waits at most one signal application, never a scan. Closes NVIDIA#5184 Signed-off-by: Emilien Macchi <emacchi@redhat.com> --------- Signed-off-by: Emilien Macchi <emacchi@redhat.com>
Signed-off-by: Gaizka Menendez <gmenende@redhat.com>
…VIDIA#5512) Signed-off-by: Evan Lezar <elezar@nvidia.com>
NVIDIA#5431) * test(conformance): port sandbox lifecycle and API e2e tests to testkit Move the driver-agnostic sandbox lifecycle, labels, templates, settings, image, process, Landlock and upload tests from e2e/rust to the conformance suite, the Podman OCI identity test to the podman driver suite, and the doctor, proxy-preflight and WebSocket tunnel checks to crate tests. Move the live-Internet benchmark to mise run perf:live-internet. Delete the legacy files and their references. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(testkit): add CLI sandbox, exec success, details, wait_absent and test image helpers Add cx.cli_sandbox (with policy files), ExecOutput::success, cx.sandbox_details, cx.wait_absent and Engine::build_test_image, use them in the ported lifecycle tests, and replace the duplicate policy file, sandbox get and test image code in the host-gateway, oci-image and podman suites. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(conformance): use ExecOutput::success for exec checks Replace hand-written exit-code checks in the smoke, lifecycle, persistent_workspace, file_transfer and policy_advisor suites. Polls and checks that expect a failure keep reading the exit code. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(testkit): add create_ready, expect_stdout, poll_exec and deadline helpers Add SandboxBuilder::create_ready, ExecOutput::expect_stdout, Sandbox::poll_exec, Context::delete_and_expect_absent and testkit::with_deadline, and convert std::io::Error into testkit::Error so I/O results take .context directly. Use them across the conformance suite, which also gets shared template, global-setting and label-list helpers and a split on-failure restart poll. Make Engine::build_image private so callers go through build_test_image, which removes the image. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(server): drive the WebSocket tunnel test through the SDK tunnel proxy Replace the hand-written WebSocket bridge with openshell_sdk::edge_tunnel::start_tunnel_proxy, so the test covers the client half of the tunnel and its edge-token headers as well as the router. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(gateway): build the preflight report once Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): make the sandbox and output helpers uniform Move expect_listed, expect_absent, wait_absent and the CLI's sandbox get JSON (now get_json) from Context onto Sandbox, and drop delete_and_expect_absent. Give ExecOutput failure() and CliOutput expect_stdout(), so both outputs offer the same checks. Put with_deadline's description first, like poll_until. Make CliSandboxBuilder::run consume the builder like create, and rename its policy setter to policy_yaml. Update the testkit README's list of what a test can do. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): reuse YamlFile and report stdout when CLI checks fail Replace CliSandboxBuilder's private PolicyFile with testkit::YamlFile, and drop YamlFile's map_err calls now that I/O errors convert into testkit::Error. CliOutput::success now includes stdout in its failure message, like ExecOutput::success, since many commands report errors there. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): use the new sandbox helpers with the credential gating changes Add SandboxBuilder::image and CliSandboxBuilder::create_ready, so both builders set an image and wait for Ready the same way. Convert tonic::Status into testkit::Error, keeping its code, so gRPC results take .context() directly, and drop Error::from_status. Create the host-gateway sandbox with cx.sandbox(..).image(..).policy(..) instead of a hand-written CLI create. In credential gating, use create_ready and poll_exec. Drop the remaining map_err calls on I/O results in host-server-support. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(gateway): compile the proxy preflight tests only with a proxy-table driver The gateway feature matrix builds without Docker, Podman or the VM driver, where the test file has no drivers to check. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(conformance): wait for labeled sandboxes and allow a 60s on-failure restart Deleting a Docker sandbox that is still starting fails with a 409 because its supervisor volume is in use, so the labels test now waits for Ready before cleanup. The VM driver needs more than 45s to report an on-failure restart; the original e2e test allowed 60s. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): share sandbox handles and output checks across builders Context, SandboxBuilder, Sandbox and CliSandboxBuilder now share one Handles value (clients, CLI, cleanup), so CliSandboxBuilder owns its state like SandboxBuilder instead of borrowing the context. Both builders are #[must_use] at the type level, and CliSandboxBuilder::image and policy_yaml take impl Into<String> like SandboxBuilder::image. ExecOutput and CliOutput run success, failure and expect_stdout through one shared check, so their messages match and failure() also reports stderr. ExecOutput.command is renamed description, since it names the exec and its sandbox. CliOutput::message is renamed report_text. serde_json and serde_yml errors convert into testkit::Error, which removes the last map_err(Error::from_error) calls. expect_listed uses let-else instead of ensure plus expect. kubernetes-updates uses create_ready. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * refactor(testkit): tighten error helpers and document test scope Read errors in gateway.rs and the image test use .context(..) now that the error types convert. Error::from_error and from_display are crate-private. Document why the global-setting helpers take &Cli, why the template test names and adopts its sandbox by hand, SandboxBuilder::restart_policy, and that the --no-keep test checks only gateway-visible removal. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(e2e): keep edge tunnel and ephemeral cleanup e2e until their ports Restore edge_tunnel_e2e with the --enable-websocket-tunnel flag in the Docker and Podman gateway wrappers, and ephemeral_cleanup with its Podman and VM lane entries. Both check what conformance cannot: the tunnel flag's wiring through a real gateway, and that drivers remove their resources. They move in later batches. Remove the in-process WebSocket tunnel server test and its openshell-sdk dev-dependency, which checked the listener but not the flag wiring. The --no-keep conformance test now says the driver-resource check stays in ephemeral_cleanup. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * chore(tasks): drop the live-Internet perf benchmark port The two live-Internet benchmarks were manual, needed the live Internet, and nothing referenced them. Remove the Python port and its perf:live-internet task instead of keeping it. NVIDIA#5472 tracks whether to bring one back. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(e2e): restore the docker proxy settings insertion unit test proxy_settings_are_inserted_before_nested_driver_tables covers insert_docker_driver_settings, which the remaining corporate proxy tests still use. It was removed with the incoherent-configuration test by mistake. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(conformance): ignore the on-failure restart test until NVIDIA#5507 On the VM driver the replacement sandbox never starts: the preserved overlay disk fails e2fsck, and the sandbox stays in Starting. The test fails on every VM lane. Ignore it with a link to NVIDIA#5507, which also tracks removing the ignore. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * test(e2e): keep the on-failure restart and global settings tests in e2e/rust Restore `on_failure_policy_replaces_runtime_and_preserves_workspace` in `sandbox_lifecycle.rs` and `settings_management.rs` as they were, and drop their conformance counterparts. The on-failure restart test ran on the Docker and Kubernetes lanes but never on VM or Podman. Ported to conformance it fails on every VM row (NVIDIA#5507), and an `#[ignore]` there removes the coverage it gave Docker and Kubernetes. The global settings test changes a setting for the whole gateway, so it can't share a conformance run with other tests until the runner has a test group for it (NVIDIA#5295). `sandbox_setting_round_trip` stays in conformance. Signed-off-by: Evan Lezar <elezar@nvidia.com> * test(conformance): only read the global setting in the settings round trip sandbox_setting_round_trip deleted the gateway-wide ocsf_json_enabled value and restored it on cleanup, which changes every sandbox on a shared gateway while it runs. Fail with a hint when a global value is set instead, as policy_advisor does. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> * docs(conformance): point the global settings test at NVIDIA#5527 Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> --------- Signed-off-by: Matthew Grossman <mgrossman@nvidia.com> Signed-off-by: Evan Lezar <elezar@nvidia.com> Co-authored-by: Evan Lezar <elezar@nvidia.com>
* fix(deps): replace yanked chacha20 in governance example Use chacha20 0.10.2 instead of the yanked 0.10.1 release in the standalone governance interceptor lockfile. The patch corrects an SSE4.1 intrinsic used by the SSE2 RNG backend. Validation: 33 governance example tests, Clippy with warnings denied, formatting, and the locked advisory scan pass. Signed-off-by: Adrien Langou <alangou@nvidia.com> * fix(deps): replace yanked yoke-derive in content guard Resolve the cargo-deny yanked-release warning in the standalone content-guard lockfile with yoke-derive 0.8.4. The existing yoke constraint accepts this patch without other dependency changes. Validation: 22 nextest tests, Clippy with warnings denied, formatting, and the locked advisory scan pass. Signed-off-by: Adrien Langou <alangou@nvidia.com> --------- Signed-off-by: Adrien Langou <alangou@nvidia.com>
…#5522) * test(network): cover private literal destination grants Signed-off-by: Adrien Langou <alangou@nvidia.com> * fix(network): require explicit grants for private IP literals Signed-off-by: Adrien Langou <alangou@nvidia.com> --------- Signed-off-by: Adrien Langou <alangou@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated Sync from main to stable
This PR automatically syncs the
mainbranch to thestablebranch by opening a pull request frommainintostable.Sync Summary
Latest commit:
b69ce7780Merge remote-tracking branch 'upstream/main'Total commits to sync: 570
Source:
https://github.com/red-hat-data-services/OpenShell.git@mainTarget:
https://github.com/red-hat-data-services/OpenShell.git@stablePR head:
mainCommits to be synced
b69ce7780Merge remote-tracking branch 'upstream/main'0d36c0e91Merge remote-tracking branch 'upstream/main'fc9288a23Merge remote-tracking branch 'upstream/main'8606ee3b1fix(review): accept SDK language topics without a parent (fix(review): accept language-only SDK classifications NVIDIA/OpenShell#5516)bde0ebc1cfeat(skills): extract issue and PR labeling (feat(skills): extract issue and PR labeling NVIDIA/OpenShell#5355)f56384b30chore(review): support independent SDK language reviewer topics (chore(review): support independent SDK language reviewer topics NVIDIA/OpenShell#5503)1f47ead2fMerge remote-tracking branch 'upstream/main'3a0e3f605Merge remote-tracking branch 'upstream/main'0d627ed59fix(deps): update hickory-proto for DNS parser advisories (fix(deps): update hickory-proto for DNS parser advisories NVIDIA/OpenShell#5513)4d777135aMerge remote-tracking branch 'upstream/main'7a9b19a10feat(skills): add omit-needless-words review skill (closes feat(skills): add omit-needless-words review skill NVIDIA/OpenShell#5496) (feat(skills): add omit-needless-words review skill NVIDIA/OpenShell#5498)f027ff7f2chore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc docker tag to v9.9 (chore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc docker tag to v9.9 (main) #128)7d43e46b9chore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.9-1793849542 (chore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.9-1793849542 (main) #126)e3e7dd83dMerge remote-tracking branch 'upstream/main'6a62690adsync pipelineruns with konflux-central - dc20ab49ad065dccMerge remote-tracking branch 'upstream/main'af9f203aeMerge remote-tracking branch 'upstream/main'86acdf985feat(review): automatically label new PRs for accepted work (feat(review): automatically label new PRs for accepted work NVIDIA/OpenShell#5260)e3edb848cfix(build): resolve sibling script imports during typechecking (fix(build): resolve sibling script imports during typechecking NVIDIA/OpenShell#5490)91703773fMerge remote-tracking branch 'upstream/main'158058908test(features): add CDI injection scenarios for Docker and Podman (test(features): add CDI injection scenarios for Docker and Podman NVIDIA/OpenShell#4461)7d9303690docs(security): clarify audit behavior and runtime guarantees (docs(security): clarify audit behavior and runtime guarantees NVIDIA/OpenShell#5452)fb1d0e340fix(deps): update rustix and refresh the RSA advisory exception (fix(deps): update rustix and refresh the RSA advisory exception NVIDIA/OpenShell#5485)9a2a6d096chore(sdk/go)!: remove the Go SDK, now at github.com/ai-openshell/sdk-go (chore(sdk/go)!: remove the Go SDK, now at github.com/ai-openshell/sdk-go NVIDIA/OpenShell#5461)3b51f0292Merge remote-tracking branch 'upstream/main'13e10405cMerge pull request fix(ci): refresh e2e tool artifact checksums opendatahub-io/openshell#178 from opendatahub-io/codex/fix-e2e-uv-checksums7b219323cfix(ci): refresh e2e tool artifact checksumsdceb984b7chore(deps): refresh rpm lockfiles [SECURITY] (chore(deps): refresh rpm lockfiles [SECURITY] #124)4b1df65f1Merge remote-tracking branch 'upstream/main'314c51ca0Merge remote-tracking branch 'upstream/main'60b8d900fMerge remote-tracking branch 'upstream/main'777ecfdcbchore(deps): refresh rpm lockfiles [SECURITY] (chore(deps): refresh rpm lockfiles [SECURITY] #120)f1ce69307fix(server): free deleted sandboxes' buffers on every replica (fix(server): free deleted sandboxes' buffers on every replica NVIDIA/OpenShell#5460)788bbdc93feat(helm): separate metrics enablement from listener port (feat(helm): separate metrics enablement from listener port NVIDIA/OpenShell#5478)ecd87dfa0Merge remote-tracking branch 'upstream/main'e6672882fMerge remote-tracking branch 'upstream/main'4ad3ce9dbfix(helm): validate gateway listener ports (fix(helm): validate gateway listener ports NVIDIA/OpenShell#3179)5f468211afeat(cli): add --output to workspace create and get (feat(cli): add --output to workspace create and get NVIDIA/OpenShell#4620)89fcc523dfix(supervisor): bound dedicated SIGTERM shutdown and stop recovering a lost boundary (fix(supervisor): bound dedicated SIGTERM shutdown and stop recovering a lost boundary NVIDIA/OpenShell#5342)31f230ed6fix(tui): cycle to real workspaces instead of a phantom "default" (fix(tui): cycle workspace scope to a known workspace, not a hardcoded "default" NVIDIA/OpenShell#4126)49eb459e7Merge remote-tracking branch 'upstream/main'f57dc4e28Merge remote-tracking branch 'upstream/main'bfa68b208fix(server): keep completed stops stable across delayed driver watches (fix(server): keep stopped sandboxes stopped after late driver updates NVIDIA/OpenShell#5204)0603b15d1chore(build): lint raw TcpStream::connect (chore(build): lint raw TcpStream::connect NVIDIA/OpenShell#5280)d2192b883test(e2e): join shared-supervisor-ha helm overlays without image-tag syntax (test(e2e): join shared-supervisor-ha helm overlays without image-tag syntax NVIDIA/OpenShell#5447)22209de75feat(compute)!: configure provisioning and bootstrap timeouts (feat(compute)!: configure provisioning and bootstrap timeouts NVIDIA/OpenShell#5321)d33c2e0a5Merge remote-tracking branch 'upstream/main'1ba385ba7Merge remote-tracking branch 'upstream/main'f3d976590fix(tmachine): restore package-installed Podman qualification (fix(tmachine): restore package-installed Podman qualification NVIDIA/OpenShell#4289)9ef1784f2Merge pull request chore(deps): update dependency nextest-rs/nextest to v0.9.151 opendatahub-io/openshell#165 from opendatahub-io/konflux/mintmaker/main/nextest-rs-nextest-0.x1545dd9faMerge pull request chore(deps): update quay.io/openshift-release-dev/ocp-release docker tag to v4.22.22 opendatahub-io/openshell#166 from opendatahub-io/konflux/mintmaker/main/quay.io-openshift-release-dev-ocp-release-4.x0df266e04Merge pull request chore(deps): update dependency astral-sh/uv to v0.13.7 opendatahub-io/openshell#167 from opendatahub-io/konflux/mintmaker/main/astral-sh-uv-0.x6715c6029Merge pull request chore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc:9.8 docker digest to d638f3b opendatahub-io/openshell#171 from opendatahub-io/konflux/mintmaker/main/registry.redhat.io-ubi9-ubi-minimal-pqc-9.8f0e84c8b1chore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.8-1794184566 (chore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.8-1794242680 opendatahub-io/openshell#172)953f4ac40Merge pull request chore(deps): update registry.access.redhat.com/ubi9/ubi docker tag to v9.8-1794186345 opendatahub-io/openshell#173 from opendatahub-io/konflux/mintmaker/main/registry.access.redhat.com-ubi9-ubi-9.x7f507155fMerge remote-tracking branch 'upstream/main'da9defe65CARRY: docs(odh): add NVIDIA-to-ODH merge sync stepsa9512a27fMerge pull request Upstream to ODH sync opendatahub-io/openshell#164 from ChristianZaccaria/odh-sync-5nov20260cb41d040Merge remote-tracking branch 'upstream/main' into odh-sync-5nov20267873aaa3bfix(server): budget SSH relays and service forwards separately (fix(server): budget SSH relays and service forwards separately NVIDIA/OpenShell#5063)eee2b9fd5test(conformance): port the credential gating e2e tests (test(conformance): port the credential gating e2e tests NVIDIA/OpenShell#5318)f62fa131efeat(gateway-interceptors): intercept sandbox start, stop, suspend, and resume (feat(gateway-interceptors): intercept sandbox start, stop, suspend, and resume NVIDIA/OpenShell#5173)be23fbc7dfix(gateway): warn in preflight when unauthenticated users are allowed (fix(gateway): warn in preflight when unauthenticated users are allowed NVIDIA/OpenShell#5105)c8fa729d5chore(deps): drop unused regorus yaml feature (chore(deps): drop unused regorus yaml feature NVIDIA/OpenShell#4882)cfb45c038fix(install): silence awk escape warnings in RPM asset lookup (fix(install): silence awk escape warnings in RPM asset lookup NVIDIA/OpenShell#4845)28407487afix(server): allocate sandbox stream buffers lazily and share events (fix(server): allocate sandbox stream buffers lazily and share events NVIDIA/OpenShell#5408)c4a158c4dfix(cli): fail status for an unregistered named gateway (fix(cli): fail status for an unregistered named gateway NVIDIA/OpenShell#5379)4453e51dafix(docker): warn on unsupported Docker Engine versions (fix(docker): warn on unsupported Docker Engine versions NVIDIA/OpenShell#5369)95240116bci(e2e): build the tmachine suite bundles once per run (ci(e2e): build the tmachine suite bundles once per run NVIDIA/OpenShell#5432)4d62992b9refactor(testkit): type the cleanup hook keys (refactor(testkit): type the cleanup hook keys NVIDIA/OpenShell#5240)d87392e3cdocs(observability): document log retention per driver (docs(observability): document log retention per driver NVIDIA/OpenShell#4901)044683e07fix(sdk/ts): revoke forward tokens on close (fix(sdk/ts): revoke forward tokens on close NVIDIA/OpenShell#5079)ed2d52245fix(server): tombstone deleted sandboxes in log and event buses (fix(server): tombstone deleted sandboxes in log and event buses NVIDIA/OpenShell#5428)569a5a052feat(kubernetes): configure supervisor pod RuntimeClass (feat(kubernetes): configure supervisor pod RuntimeClass NVIDIA/OpenShell#5349)569f03a1dfix(vm): record kernel provenance in runtime tarballs (fix(vm): record kernel provenance in runtime tarballs NVIDIA/OpenShell#4650)3228860c9docs: update OpenShell naming and announce CNCF transition (docs: update OpenShell naming and announce CNCF transition NVIDIA/OpenShell#5441)4a009c384fix(cli): use a POSIX path probe in sandbox download (fix(cli): use a POSIX path probe in sandbox download NVIDIA/OpenShell#5420)9977edce9fix(cli): renew client credentials OIDC tokens (fix(cli): renew client credentials OIDC tokens NVIDIA/OpenShell#4389)baf7d3445chore(deps): refresh rpm lockfiles [SECURITY] (chore(deps): refresh rpm lockfiles [SECURITY] #115)7e960b31achore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.8-1794242680 (chore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.8-1794242680 #114)a5094bfabchore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc:9.8 docker digest to d638f3b (chore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc:9.8 docker digest to d638f3b #113)1b424ba2cchore: reconcile component repo PipelineRuns on main (chore: reconcile component repo PipelineRuns on main #111)93bde2fd9chore(deps): update registry.access.redhat.com/ubi9/ubi docker tag to v9.8-1794186345e28c12fe3chore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc:9.8 docker digest to d638f3b4f7901865chore(python): deprecate the Python SDK in core (chore(python): deprecate the Python SDK in core NVIDIA/OpenShell#5350)9cf85f9f3ci(e2e): start Linux and tmachine jobs without waiting on macOS or package builds (ci(e2e): start Linux and tmachine jobs without waiting on macOS or package builds NVIDIA/OpenShell#5270)9e004a746ci(e2e): run the tmachine conformance rows with the prebuilt openshell-conformance (ci(e2e): run the tmachine conformance rows with the prebuilt openshell-conformance NVIDIA/OpenShell#5332)00222c281fix(python): add opt-in sandbox delete timeout (fix(python): add opt-in sandbox delete timeout NVIDIA/OpenShell#4697)56d235056fix(network): suppress SigV4 signing diagnostics (fix(network): suppress SigV4 signing diagnostics NVIDIA/OpenShell#4558)5e3cb25dafix(deps): update h2 and defer SQLite until SQLx release (fix(deps): update h2 and defer SQLite until SQLx release NVIDIA/OpenShell#5392)f884776d9Merge remote-tracking branch 'upstream/main'a28f59b7efix(sandbox): document unsafe blocks and enforce SAFETY comments with clippy (fix(sandbox): document unsafe blocks and enforce SAFETY comments with clippy NVIDIA/OpenShell#5232)68b610420chore(server): remove unused registration hooks (chore(server): remove unused registration hooks NVIDIA/OpenShell#5382)25997cbf7Merge pull request CARRY: test(odh): run supported Python gateway e2e on OpenShift opendatahub-io/openshell#161 from opendatahub-io/test/python-ocp-e2e/gmenher9b803f70cCARRY: test(odh): run supported Python gateway e2e on OpenShift9525bec0ddocs: correct security boundaries and runtime guidance (docs: correct security boundaries and runtime guidance NVIDIA/OpenShell#5331)e46f2b956chore(deps): refresh rpm lockfiles [SECURITY] (chore(deps): refresh rpm lockfiles [SECURITY] #110)0052e73a9chore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.8-1794184566 (chore(deps): update registry.access.redhat.com/ubi9/nodejs-24-minimal docker tag to v9.8-1794184566 #109)ef2cbb020chore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc:9.8 docker digest to ddaf8f8 (chore(deps): update registry.redhat.io/ubi9/ubi-minimal-pqc:9.8 docker digest to ddaf8f8 #108)bbbedcc41chore(deps): refresh rpm lockfiles [SECURITY] (chore(deps): refresh rpm lockfiles [SECURITY] #107)Merging
GitHub automerge is enabled for this pull request once required checks pass.