Repository navigation
feat(#7481): allow OpenAI-only repositories without GCP configuration - #7718
Conversation
…l in github setup A repository that runs every agent on GPT — through OpenAI Workload Identity Federation (fullsend-ai#6689, ADR 0092) or the FULLSEND_OPENAI_API_KEY secret (fullsend-ai#7295) — still had to provide a GCP project and WIF provider it never used, because per-repo setup treated the GCP pair as the definition of "inference is configured". Add "openai" to ValidProviders so inference.provider can declare a GPT-only repository (vertex stays the default; the runner keeps picking the provider per model — this value only drives install-time checks). In `fullsend github setup <owner/repo>`, when the effective provider is openai and neither --inference-project nor --inference-wif-provider is supplied, skip the GCP secret lookups and write no GCP secret; the values were previously written unconditionally, which under openai would have stored an empty FULLSEND_GCP_PROJECT_ID. Supplying one GCP value under openai keeps the partial-pair error. Provider openai must have a route: the --openai-* trio in the composed config or an existing FULLSEND_OPENAI_API_KEY secret, otherwise setup refuses and names both, the same way a Vertex install refuses without a project. The two GCP "is required" errors now point at --inference-provider openai. A partial inference.openai block in an existing layer is refused as well, even when the secret exists: the runner errors on a partial trio instead of falling back to the key, so the secret cannot make it a route (Codex review). Tests that used "openai" as the example of an invalid provider now use "bedrock". Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
…ge, repos install probe/converge/status defined "inference is configured" as the GCP pair (requiredSecrets), so a repository on the openai route was reported incomplete and converge insisted on --inference-project for it. Add InferenceRoute, read from the committed per-repo config layers (.fullsend/config.yaml over config.base.yaml): provider, whether the OpenAI WIF trio is complete, and whether any layer exists. Required secrets become route-aware: vertex keeps requiredSecretsForForge; openai needs FULLSEND_OPENAI_API_KEY on GitHub unless the trio is committed, and on GitLab (no WIF there) the project's own OPENAI_API_KEY variable plus the forge token. That GitLab variable is probed only — never provisioned, never deleted — and a test pins it out of every uninstall list. requiredSecretsForForge stays the managed superset for orphan detection and uninstall. ProbeComponents takes a variadic ProbeOption; converge passes WithDefaultProvider(manifest provider) for a repo with no config yet, so a fresh install probes the secrets it is about to need. An openai-route repo with no GCP flags skips WIF derivation and the project requirement, but converge refuses it when neither the trio nor the OpenAI secret exists, the same way github setup does. GCP flags passed for the batch still apply to openai repos (both routes). An unparsable config layer is an error, not a silent vertex default. A partial inference.openai block on the openai route is an error from ProbeInferenceRoute, so probe, status and converge cannot call such a repository healthy on the strength of the secret. Converge resolves the route once and hands it to ProbeComponents through WithInferenceRoute, so its own decisions and the component probe read the same value; status passes the manifest provider through WithDefaultProvider the same way converge does. A repair that regenerates config.yaml (missing workflow file) keeps a committed openai route instead of reverting the repository to vertex when its manifest entry predates inference_provider; vertex repositories still get no provider line. (Codex review.) repos.yaml gains inference_provider (defaults and per-repo, validated like runtime, "none" stops the chain); ResolvedConfig and InstallConfig carry it and BuildScaffoldFiles writes inference.provider only when set, so vertex installs are unchanged. `repos install --inference-provider` records it on repos the command adds. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
…GCP secrets
The setup-gcp composite action declared gcp_wif_provider required and
ran google-github-actions/auth unconditionally, and every reusable
workflow declared the GCP secret pair `required: true`. A repository on
the openai route (inference.provider openai) has neither secret, so its
agent jobs failed at Google auth before the runner started, and
prepare-sandbox-credentials.sh would have read an empty
GOOGLE_APPLICATION_CREDENTIALS.
Make gcp_wif_provider optional and guard all four setup-gcp steps with
`if: inputs.gcp_wif_provider != ''`; declare both GCP secrets
`required: false` in reusable-{dispatch,triage,code,review,fix,retro,
prioritize}.yml. Callers are unchanged: the scaffold shims keep passing
the secrets, which are simply empty on such a repository. The runner's
own GCP refresh loop is already conditional on FULLSEND_GCP_OIDC_URL.
A first "Check GCP inputs" step fails fast when exactly one of the two
inputs is populated: a Vertex repository missing one secret is refused
there, where the message names both, rather than at the first model
call — and no ambient runner credential can stand in for the missing
one (Codex review). Both empty is the openai route and skips the rest.
TestGCPSecretsOptionalForOpenAIRepos pins both halves so a future
`required: true` or an unguarded step is caught.
Assisted-by: Claude
Signed-off-by: Wayne Sun <gsun@redhat.com>
…al GCP settings - OpenAI guide: new section D, "A repository with no GCP project at all" — how to declare the openai route with github setup, what setup refuses, how probe/status/install judge it, and that the runner still selects the provider per model. - getting-inference: point GPT-only repositories past the GCP steps. - repo-management: partial-secret section scoped to the Vertex route; new section for openai repositories including the refusal message and the GitLab OPENAI_API_KEY ownership rule. - CLI reference: --inference-provider on github setup and repos install; defaults.inference_provider in the manifest key table. - layered-config, operations and infrastructure references: the GCP secrets are not needed / not written for openai repositories; the OpenAI secret is the required one there. Assisted-by: Claude Signed-off-by: Wayne Sun <gsun@redhat.com>
Keep a committed OpenAI WIF trio when scaffolds are regenerated, carry the selected provider into preset overlays, and implement the documented manifest default editor key. Assisted-by: Codex Signed-off-by: Amos Mastbaum <amastbau@redhat.com>
Signed-off-by: Amos Mastbaum <amastbau@redhat.com> Assisted-by: Codex
Signed-off-by: Amos Mastbaum <amastbau@redhat.com> Assisted-by: Codex
E2E tests are runningAuthorization passed for this commit. See the E2E Tests workflow for results. |
PR Summary by QodoAllow repositories to install without GCP and route inference per agent
AI Description
Diagram
High-Level Assessment
Files changed (33)
|
Code Review by Qodo
1.
|
Allow fleet harnesses to opt a required host file into an OpenAI-only empty-source skip. Keep required-file errors for Vertex runs and unannotated mounts, including Scribe Drive credentials. Signed-off-by: Amos Mastbaum <amastbau@redhat.com> Assisted-by: Codex
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 9:20 PM UTC · Completed 9:47 PM UTC Commit: Runtime: pi · Model: sonnet → claude-sonnet-5 · Effort: high · Cost: $13.45 |
Site previewPreview: https://ff2deee8-site.fullsend-ai.workers.dev Commit: |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Risk Assessment: elevated (3/5) DetailsLarge diff spanning 47 files with modifications to 8 protected CI workflow paths and high historical churn in touched packages elevate risk, balanced by an opt-in configuration design and complete alignment with the linked issue. |
ReviewFindingsHigh
Medium
Low
Next steps:
|
There was a problem hiding this comment.
Note: The following review comments could not be posted on the diff (GitHub returned 422) and are included here instead:
internal/repos/probe.go(file-level): Line 138 · [high] regression
This PR's conflict resolution against current main dropped GitLab scaffold/probe/converge/uninstall behavior that already landed on main, and every test that pins it — and it introduces a guaranteed compile failure. Verified independently: target-repo/internal/repos/uninstall.go (current main) defines gitlabAuxiliaryScriptPaths(), the package-level var gitlabRetiredScaffoldPaths, and the constants gitlabInstallCLIScriptPath/gitlabPollJobScriptPath/gitlabAgentJobScriptPath. None of these exist anywhere in this PR's changed files (grep -rn across /sandbox/workspace/pr-head/ returns zero hits). internal/repos/remote_scaffold_test.go:113 (unchanged by this PR, package repos) references gitlabRetiredScaffoldPaths, and internal/repos/uninstall_test.go:651-653 (also unchanged) references gitlabInstallCLIScriptPath/gitlabPollJobScriptPath/gitlabAgentJobScriptPath. Since those two test files are not part of this diff, they retain their current-main content when this PR merges — and the repos test package will fail to compile. Beyond the compile break, probe.go (line 138) still walks only gitlabTrustScriptPath/gitlabRoleTokenScriptPath instead of the fuller gitlabAuxiliaryScriptPaths() set, and no longer uses readWorkflowMarker; converge.go's orphan handling only warns instead of deleting known-retired GitLab paths. GitLab-specific tests present on current main (TestProbeComponents_GitLab_MissingExtractedJobScripts, TestProbeComponents_GitLab_ReadsPipelineMarker, TestProbeComponents_GitLab_FallsBackToLegacyDispatchMarker, TestProbeComponents_GitLab_PrefersPipelineMarkerOverLegacy, TestProbeComponents_GitLab_LegacyDispatchOnly, TestConverge_GitLab_RepairsMissingExtractedJobScripts, TestConverge_GitLab_RemovesObsoleteDispatchFile, TestConverge_GitLab_RemovesObsoleteDispatchFile_DryRun, TestConverge_GitLab_KeepsDispatchStubWhenPinnedRefWrapperStillIncludesIt) are absent from this PR's probe_test.go/converge_test.go. This contradicts the PR's own testing checklist, which claims go test ./internal/repos ... and make go-test passed.
Suggested fix: Re-merge current main into probe.go, uninstall.go, converge.go, probe_test.go, converge_test.go, and check whether remote_scaffold_test.go/uninstall_test.go need matching updates. Keep both sides: restore readWorkflowMarker, gitlabAuxiliaryScriptPaths, gitlabRetiredScaffoldPaths (and the converge delete path that consumes it), the additional GitLab helper-script constants/paths, and every GitLab probe/converge test from main — then re-apply only the OpenAI route changes (requiredSecretsForRoute, WithInferenceRoute, and the new openai-route tests) on top.
internal/repos/converge.go(file-level): Line 258 · [medium] logic-error
secretsPresent still hard-codes both FULLSEND_GCP_PROJECT_ID and FULLSEND_GCP_WIF_PROVIDER as the probed component list. On the openai route those GCP secrets are not probed, so hasSecrets is always false for an openai repo. An already-installed openai repo takes the component-converge path, where convergeSecrets only writes secrets that appear as missing probe components — so batch --inference-project/--inference-wif-provider flags passed during a re-converge are silently ignored for existing openai repos. This contradicts the dual-route behavior this PR's own doc update (docs/guides/getting-started/repo-management.md) describes: passing the GCP flags for the batch still applies them to such repositories. The fresh-install path (Install) is unaffected and is what the added test (TestConverge_OpenAIRoute_GCPFlagsStillApplyWhenGiven) covers, so the re-converge gap is untested.
Suggested fix: Make secretsPresent (and convergeSecrets) route-aware: treat GCP flags as an independent write set from the openai required-secret probe list, or probe leftover GCP secrets when the flags are supplied so an existing openai repo can still receive the Vertex pair on re-converge.
docs/guides/infrastructure/openai-workload-identity.md(file-level): Line 274 · [medium] stale-guide-instructions
Route C step 1 tells users to re-run fullsend github setup <owner/repo> (without --inference-provider openai) before step 3 sets FULLSEND_OPENAI_API_KEY. On a repository with no GCP credentials, that command fails today: setup requires the GCP pair unless --inference-provider openai is passed, and that flag in turn requires the OpenAI secret (or a complete inference.openai trio) to already exist. Neither Route B nor Route C states that GCP is not required — Section D in the same document does give the correct secret-first-then---inference-provider openai order, but a reader following Route C's numbered steps can easily miss it and get stuck in a chicken-and-egg sequence. This falls short of issue #7481's stated documentation goal that "the OpenAI guide's Route B/C sections say GCP is not required."
Suggested fix: In Route C, distinguish resyncing an existing (Vertex-provisioned) repo from openai-only enrollment; for the latter, point to Section D's order (set FULLSEND_OPENAI_API_KEY first, then run fullsend github setup <owner/repo> --inference-provider openai), and state explicitly in Route B/C that GCP is not required when --inference-provider openai is used.
docs/guides/infrastructure/infrastructure-reference.md(file-level): Line 415 · [medium] missing-documentation
Under the GitLab CI/CD variables table, FULLSEND_GCP_PROJECT_ID/FULLSEND_GCP_WIF_PROVIDER are still listed as required when inference is configured, with no note that they are omitted when inference_provider is openai, and OPENAI_API_KEY is described only as optional rather than as the required credential for GitLab repos on the openai route (per requiredSecretsForRoute). This PR updated the equivalent GitHub per-repo secrets section a few dozen lines above, but the GitLab section on the same page was missed.
Suggested fix: Update the GitLab CI/CD variables section to note that the GCP pair is not required when inference_provider is openai, and that OPENAI_API_KEY is required on GitLab in that case.
docs/cli/repos.md(file-level): Line 94 · [low] incomplete-cli-reference
The prose description of repos install's "Manifest add" behavior lists the per-repo flag overrides written to the manifest entry (--inference-region, --fullsend-ref, --mint-url, --allowed-remote-resources, --runtime) but omits --inference-provider, even though this PR adds that flag and it is documented in the flags table further down the same page.
Suggested fix: Add --inference-provider to the parenthetical list of per-repo overrides at line 94.
docs/guides/getting-started/repo-management.md(file-level): Line 28 · [medium] contradictory-prerequisites
The prerequisites section and installation notes still assert unconditionally that GCP WIF provisioning must be completed before running repos install, while a new section added later in the same file by this PR states that repositories using inference_provider: openai need neither GCP secret. The getting-started path now contradicts the openai-only path documented lower on the same page.
Suggested fix: Qualify the prerequisites/installation-notes language to state that GCP WIF provisioning is required only for repositories using the default Vertex inference route.
docs/cli/repos.md(file-level): Line 114 · [low] incomplete-cli-reference
The repos install flags table documents --inference-project/--inference-wif-provider without noting they are optional when the provider is openai, unlike docs/cli/github.md, which this PR updated with "(required unless the provider is openai)" wording for the equivalent flags.
Suggested fix: Update the --inference-project/--inference-wif-provider descriptions in the repos install flags table to note they are required only for the vertex route.
docs/guides/getting-started/repo-management.md(file-level): Line 137 · [low] incomplete-manifest-reference
The prose describing the none sentinel in repos.yaml lists fullsend_ref, mint_url, mint_mode, config_base.source, config_base.sha256 but omits runtime and the newly added inference_provider, both of which accept the none sentinel per manifest.go/manifest_test.go.
Suggested fix: Add runtime and inference_provider to the list of string fields supporting the none sentinel.
waynesun09
left a comment
There was a problem hiding this comment.
Review with three models (Codex, Claude, Grok via pi) plus a local run. Only MEDIUM+ findings not already raised by qodo or the review bot are posted inline.
Local verification at 1d03f99 (agents#1502 harnesses, every GCP variable unset): fullsend run triage --runtime pi --model openai/gpt-5.6-luna created the sandbox, the agent exited 0 and validation passed. So the no-GCP OpenAI path works end to end in the runner, which covers part of the unchecked "real agent run" box. The same env with --runtime claude fails late; see the harness.go thread.
Not inline (the file is outside the diff), MEDIUM: fullsend agent new (internal/agentnew/render.go buildHarness) still emits {Src: "${GOOGLE_APPLICATION_CREDENTIALS}", Dest: "/tmp/.gcp-credentials.json"} without OptionalForOpenAI. So a harness scaffolded for an openai-only repo fails the same way the fleet harnesses did before agents#1502. Consider setting the flag there, plus a render test.
qodo's leftover partial GCP pair (#1/#2), the batch-flag bypass of the OpenAI credential check (#3) and the preset-route timing (#4) still reproduce at this head, so they are not re-posted. qodo's drift-revert finding (#5) no longer applies, because CheckFileContentDrift skips both config layers.
…ps audience Workload identity providers provisioned by fullsend allow the audiences https://iam.googleapis.com/<provider> and fullsend-mint. The runner asked GitHub for a token with the //iam.googleapis.com/ form, which STS rejects. Request the https form, keep the scheme-relative form for the STS exchange and the credential file, and carry the https form in FULLSEND_GCP_OIDC_URL for the refresh loop. Add an optional OnSubjectToken callback so the caller can mask the fetched token in its log. The test's fake OIDC endpoint now refuses audiences outside the provider's allowed list, so the old request form fails the test. Assisted-by: Claude (fix, review) Signed-off-by: Wayne Sun <gsun@redhat.com>
On GitHub Actions, fullsend run now picks Vertex credentials in this order:
both FULLSEND_GCP_PROJECT_ID and FULLSEND_GCP_WIF_PROVIDER set prepares
WIF and overrides GOOGLE_APPLICATION_CREDENTIALS; neither set keeps an
existing credential file from an earlier step; one set is an error. A
kept file must be non-empty JSON with a type, and an external_account
file must read its token from credential_source.file: a URL source is
unreachable from the sandbox and its headers would copy the runner's
request token into it. The run prints which source it used.
An OpenAI parent can dispatch Vertex sub-agents, so it also prepares WIF
when both inputs are set; a failure there is a warning. The dummy
runtimes resolve to a "none" provider and need no GCP inputs. The
Vertex-parent path stays fail-closed.
Runtime resolution and credential preparation now run before harness
env validation and expansion. A harness whose host_files require
${GOOGLE_APPLICATION_CREDENTIALS} failed validation on every GitHub
Actions Vertex run, because the workflow no longer sets it before
fullsend run starts.
The fetched and refreshed GitHub OIDC tokens are masked in the Actions
log. validateVertexGCPCredentials rejects an empty file.
prepareGitHubWIF is a package variable so runAgent tests can stub the
exchange.
Functional tests no longer receive the GCP inputs, so they keep using
their prepared credential file.
Assisted-by: Claude (fix, review)
Signed-off-by: Wayne Sun <gsun@redhat.com>
Describe how fullsend run picks Vertex credentials on GitHub Actions, what local and GitLab runs need, and that an OpenAI run with the GCP inputs still gets credentials for Vertex sub-agents. Align the OpenAI workload identity guide and the workflow contract note with that. Assisted-by: Claude (fix, review) Signed-off-by: Wayne Sun <gsun@redhat.com>
Assisted-by: Claude (fix, review) Signed-off-by: Wayne Sun <gsun@redhat.com>
… credentials Assisted-by: Claude (fix, review) Signed-off-by: Wayne Sun <gsun@redhat.com>
A run whose parent does not use Vertex now clears GOOGLE_APPLICATION_CREDENTIALS when the file fails validation, so an optional host file cannot copy a URL-source credential into the sandbox. The validator caps the file size and checks impersonation sources. Dummy runtimes skip WIF. Assisted-by: Claude (fix, review) Signed-off-by: Wayne Sun <gsun@redhat.com>
…file
A non-Vertex run whose required ${GOOGLE_APPLICATION_CREDENTIALS} mount has
no file now fails before the pre-script. The credential check follows
source_credentials only for impersonation files.
Assisted-by: Claude (fix, review)
Signed-off-by: Wayne Sun <gsun@redhat.com>
…e set Behaviour tests run dummy agents on fleet harnesses whose GCP credential mount is required. Skipping WIF for dummy runs left that mount empty and failed the triage scenarios. Dummy runs now follow the OpenAI-parent rule. Assisted-by: Claude (fix, review) Signed-off-by: Wayne Sun <gsun@redhat.com>
|
@amastbau I'm done; the branch is yours again. Head is fbfd01d: the review fixes, a merge of current main (the branch still pinned OpenShell 0.0.116), and one follow-up so behaviour-test dummy runs keep WIF. The PR description lists the live runs on Vertex and OpenAI, with both claude and pi. Behaviour, e2e and CI are green. functional-tests is failing in its "Create ephemeral repo and fixture" hook before any case runs; other PRs hit the same failure right now. agents#1502 also needs current agents main merged before it can land. Its |
Summary
AISDLC-127 / #7481: repositories can now install without Vertex credentials, and each agent run selects inference from that agent's effective runtime and model. Agents in one repository may use different providers.
github setupandrepos installaccept no GCP inference inputs and write no GCP credential secrets. When GCP inputs are provided, the existing Vertex setup remains.repos statusaccepts both secrets absent but reports a partial pair as drift.fullsend runhandles the selected agent. On GitHub Actions, Google credentials are chosen before harness validation and the pre-script:https://iam.googleapis.com/<provider>audience that provisioned providers allow.GOOGLE_APPLICATION_CREDENTIALSfile after checking it; a URL-sourceexternal_accountfile is rejected.host_files[].optionalfor six inference harnesses. Scribe is unchanged.Documentation
This PR updates the GitHub/GitLab setup guides, repo-management and inference guides, CLI references, and workflow-contract notes. Docs are included here; no follow-up docs PR is planned. The guides explain that installing without GCP does not make a Vertex agent runnable without its credentials.
Verification
make lintand commit hooks passed.make go-testpassed with race detection; overall statement coverage: 82.9%. Changed functions:runGitHubSetupPerRepo85.4%,resolveInferenceReuse100%,Converge83.0%,ProbeComponents95.7%.New tests cover GitHub and GitLab installation without GCP, status with both secrets absent, partial-pair drift, and existing Vertex setup behavior.
Live runs at 75e9279 (and its merge parent); fbfd01d then restores WIF for dummy runtimes, which the behaviour suite needs. The GitHub Actions runs used a disposable per-repo install whose WIF provider came from
fullsend inference provision; the local runs used OpenShell 0.1.2:opusxai-vertex/xai/grok-4.6openai/gpt-5.6-luna, Vertexsubagents.defaultopenai/gpt-5.6-luna, fleet harness on agents mainhost_files[1]: GOOGLE_APPLICATION_CREDENTIALS is empty; mark the mount optional or provide a credential file(needs agents#1502)openai/gpt-5.6-luna, agents#1502 harnessxai-vertex/xai/grok-4.6At 2f113b6 the same Actions install failed with
host variable GOOGLE_APPLICATION_CREDENTIALS is not set. With the OIDC audience that head sent, the provisioned provider would also have rejected the token exchange.Generated with Codex