Add conformance test for tools/call#2
Merged
Conversation
coding-bobo
added a commit
to coding-bobo/conformance
that referenced
this pull request
May 21, 2026
Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable, tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario (PR modelcontextprotocol#2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates cross-app-access.ts to use it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
coding-bobo
added a commit
to coding-bobo/conformance
that referenced
this pull request
May 21, 2026
Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable, tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario (PR modelcontextprotocol#2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates cross-app-access.ts to use it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
coding-bobo
added a commit
to coding-bobo/conformance
that referenced
this pull request
Jun 3, 2026
Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable, tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario (PR modelcontextprotocol#2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates cross-app-access.ts to use it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pcarleton
pushed a commit
that referenced
this pull request
Jul 2, 2026
…1933) (#268) * feat(auth): add JWT-bearer helper for WIF client conformance (SEP-1933) Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable, tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario (PR #2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates cross-app-access.ts to use it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(auth): add JWT-bearer helper for WIF client conformance (SEP-1933) Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable, tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario (PR #2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates cross-app-access.ts to use it. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(auth): add WIF JWT-bearer scenario and negative tests (SEP-1933) Adds the auth/wif-jwt-bearer client conformance scenario using the RFC 7523 JWT-bearer grant (urn:ietf:params:oauth:grant-type:jwt-bearer). The scenario pre-signs valid, wrong-audience, and expired JWTs on start() to simulate cloud workload identity tokens. The conformance AS verifies the assertion and emits per-class checks (wif-assertion-verified, wif-assertion-missing, wif-assertion-audience, wif-assertion-expired, wif-assertion-malformed). Broken example clients exercise the missing-assertion and wrong-audience failure paths. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: apply prettier formatting to wif-jwt-bearer.ts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(auth): add expired-assertion negative test; clarify wif-jwt-bearer comments - Add runWifJwtBearerExpiredAssertion + auth-test-wif-expired-assertion.ts to exercise the wif-assertion-expired check path (was dead code) - Clarify WifAssertionVerified description to note iss is not validated - Add comment explaining clockTolerance: 5 is intentional (same-run keypair) - Add comment explaining numeric expiresIn is absolute epoch seconds Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): disable DCR, detect retries, improve getChecks description - Add WIF_CLIENT_ID constant; pre-seed provider's clientInformation() so the SDK skips Dynamic Client Registration entirely (disableDynamicRegistration on the auth server + pre-seeded client_id on the provider side) - Add failedOnce/tokenRequestReceived tracking on the scenario; reject and record wif-no-retry FAILURE if the client attempts a second token request after the first fails - Add hasAttempted guard in WifJwtBearerProvider.prepareTokenRequest() to throw on retry from the client side - Fix getChecks() sentinel description: distinguish "no request made" from "request made but verification failed" - Add client_id to context and Zod schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: prettier formatting on everything-client.ts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): let errors propagate from broken example clients The try/catch blocks were silently swallowing auth failures, so the negative tests passed purely because expectedFailureSlugs found the AS-emitted check — not because client error-surfacing was verified. allowClientError: true on the test cases handles the non-zero exit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): tag as draft, add scope-rejected check, add SEP traceability - Change specVersions from ['extension'] to [DRAFT_PROTOCOL_VERSION] and add source = { introducedIn: DRAFT_PROTOCOL_VERSION } so the scenario is reachable via --spec-version draft (extension tag excluded it from all spec-version runs) - Move registration from extensionScenariosList to draftScenariosList in index.ts to match the tag/list convention enforced by spec-version.test.ts - Add wif-assertion-scope-rejected check: AS returns invalid_scope for a valid JWT when the client requests the reserved 'wif.rejected' scope; verify client surfaces the error and does not retry - Add runWifJwtBearerScopeRejected to everything-client.ts; add optional scope param to WifJwtBearerProvider; add CLI entry point and vitest case - Add src/seps/sep-1933.yaml with requirements mapped to each check ID, covering the three deferred checks (iss, sub, jti) with exclusion rationale Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: prettier formatting Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(wif-jwt-bearer): add wif-grant-fallback check for unauthorized_client fallback Client bug class: WIF client receives unauthorized_client from JWT-bearer grant and silently switches to authorization_code instead of surfacing the error. The MCP SDK retries after UnauthorizedClientError (auth.js:152-154), calling prepareTokenRequest() a second time. WifGrantFallbackProvider exploits this by returning authorization_code params on the second call. Spec anchor: RFC 7523 §2.1 — clients MUST use the JWT-bearer grant type; silent grant-type switching hides authentication failures. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: rename unused scope param in WifGrantFallbackProvider Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(wif-jwt-bearer): use realistic SPIFFE and K8s PSAT token formats valid_jwt uses a SPIFFE JWT-SVID subject (spiffe://conformance-test.local/...) issued by a SPIRE-style issuer. wrong_audience_jwt uses a Kubernetes PSAT subject and kubernetes.io claims, mirroring how K8s projected service-account tokens look in production. Token verification and client behaviour are unchanged; the formats ground the scenario in real workload identity platforms. k8s_issuer and k8s_subject added to context and schema for external clients that want to construct their own K8s-style assertions. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: prettier formatting on wif-jwt-bearer.ts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(wif-jwt-bearer): drop informational k8s_issuer/k8s_subject from context These fields were not used by any client handler and are redundant since the K8s claims are already baked into wrong_audience_jwt. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): address pre-merge review feedback - Downgrade wif-no-retry, wif-grant-fallback, and wif-assertion-scope-rejected to WARNING; RFC 7523 is silent on client retry and grant-type switching, so FAILURE status was not traceable to spec text - Add WifRetryProvider broken client that re-sends JWT-bearer after unauthorized_client, making wif-no-retry actually exercisable as a WARNING - Drop getChecks() sentinel (non-standard pattern that overloaded the wif-assertion-verified check ID); test harness detects empty check sets - Add comment on DRAFT_PROTOCOL_VERSION workaround for runner limitation - Add comment explaining why both slash/no-slash audience forms are accepted - Tighten schema comments for audience URL constraint and ES256 literal Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): drop decorative fields and neutralise JWT fixtures - Remove issuer and subject from context schema and scenario return value; neither is validated by the AS or used by the client, making them misleading. The iss exclusion rationale is updated to explain why. - Replace SPIFFE/K8s-PSAT constants and the kubernetes.io additionalClaims with neutral values across all three JWT fixtures; no check inspects token format, so format flavour was decorative and invited false assumptions about fixture semantics. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): remove workaround comment, drop signing_algorithm, annotate retry provider - Remove self-admitted workaround comment on DRAFT_PROTOCOL_VERSION; the classification stands on its own and the comment was inviting challenge - Drop signing_algorithm from context schema and scenario return; the field was redundant once the value was fixed to ES256, and the assertion is opaque to the client regardless - Add comment to WifRetryProvider explaining it deliberately omits the hasAttempted guard so the SDK retry reaches the AS Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): export scope constants, remove unused AS field, annotate state machine - Export WIF_TRIGGER_UNAUTHORIZED_SCOPE and WIF_REJECTED_SCOPE from createWorkloadJwt.ts and import them in everything-client.ts; removes duplicate bare-string declarations that could silently diverge - Remove tokenEndpointAuthSigningAlgValuesSupported from mock AS config; the SDK does not consume this field in the JWT-bearer flow - Add comment to onTokenRequest clarifying wif-no-retry and wif-grant-fallback fire on any second request after any failure, not only post-unauthorized_client Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): address pcarleton review comments - Move all broken WIF provider classes and runners to wif-broken-clients.ts; everything-client.ts now contains only the conformant WifJwtBearerProvider - Add sentinel in getChecks() for !tokenRequestReceived to prevent a silent pass when the client never sends a request - Drop wif-assertion-scope-rejected check; the no-retry behaviour after invalid_scope is already covered by wif-no-retry, and sep-1933.yaml updated accordingly - Remove audience from context schema and scenario return; the field was unused by any client implementation - Expand scenario description to tell a client implementer what to implement - Apply suggested description fix to wif-grant-fallback (does not assume unauthorized_client was the specific error that preceded the fallback) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(wif-jwt-bearer): drop number type from expiresIn, use jose duration strings Jose supports negative duration strings ('-60s', '60 seconds ago') that produce already-expired tokens without needing absolute epoch arithmetic. Drop the number overload, update the expired_jwt call site to '-60s', and update the unit test accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
pcarleton
pushed a commit
that referenced
this pull request
Jul 10, 2026
* Add DPoP client conformance scenario + shared DPoP helpers (SEP-1932, #368) Client-anchored packaging: carries the shared DPoP foundation (dpopProof / dpopToken helpers, the createAuthServer DPoP core, and sep-1932.yaml) together with the client scenario, so it can merge first; the server (#369) and authorization-server (#370) scenarios are follow-ups that depend on it. DPoP is treated as a draft/not-yet-official feature (mirroring the WIF SEP-1933 scenario): the scenario's source is `introducedIn: DRAFT_PROTOCOL_VERSION` and it is registered in draftScenariosList (informational, not scored). It is NOT registered in EXTENSION_IDS, since the ext-auth DPoP extension is still on a branch rather than merged. Client checks (the two extension MUSTs, RFC 9449 §7.1 / §4.2-4.3): - sep-1932-client-dpop-auth-scheme — token presented with the DPoP scheme. - sep-1932-client-fresh-proof — a fresh, well-formed DPoP proof per request. The test MCP server judges the client via an opt-in DPoP middleware (dpopResourceAuth) passed through createServer's authMiddleware hook; a compliant example client (covered by the Client Draft Scenarios loop) plus deliberately- broken bearer/replay variants drive the pass/fail acceptance tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * client/dpop: split into nonce-less baseline + nonce-required scenario Server-provided nonces are optional in RFC 9449 (AS §8 "MAY", RS §9 "can also choose"), and the nonce-less flow is the common case. The single auth/dpop scenario hard-wired nonce enforcement on both the test AS and MCP server, so a plain nonce-less proof was never accepted end-to-end and a client that only does plain proofs was never exercised on its happy path. Parameterize DPoPClientScenario with `requireNonce` and register it twice: - auth/dpop (requireNonce=false) — nonce-less baseline; emits the three baseline checks (token-request-proof, dpop-auth-scheme, fresh-proof). - auth/dpop-nonce (requireNonce=true) — prior behavior; adds as-nonce and rs-nonce (five checks). Check IDs are reused (no new IDs, no sep-1932.yaml change); the two nonce checks are emitted only by auth/dpop-nonce, so traceability stays intact. Add auth-test-dpop-no-nonce.ts (nonce-incapable but otherwise compliant) asserting SUCCESS against auth/dpop — proof that a client with no nonce handling still completes DPoP when the server does not require a nonce. Baseline negatives stay on auth/dpop (nonce-independent); the two nonce negatives are re-pointed to auth/dpop-nonce. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * client/dpop: round-4 nonce-surface correctness fixes Address review findings on the auth/dpop-nonce posture: - Record the token-request proof observation BEFORE the §8 nonce challenge, so a client that is challenged but does not retry is no longer mis-reported by token-request-proof as "never completed a token request" (its proof was just validated). (#2) - Record the resource-request jti/replay on ANY valid proof (before the §9 nonce gate), so fresh-proof is never asserted on zero evidence and a client that never honours the RS nonce fails ONLY rs-nonce. (#5) - Require challengeIssued && honored for as-nonce/rs-nonce SUCCESS, closing a vacuous-pass path (a client that pre-sends the nonce is never challenged). (#3a) - Gate the AS nonce observation on the authorization_code exchange, matching recordTokenRequestProof (a refresh exchange must not satisfy §8). (#3c) - Collapse duplicate shared token-flow check IDs (token-request, pkce-*) that the §8 round-trip re-POST would otherwise emit twice. (#4) - Example client: retry the token request only on a use_dpop_nonce error code, not any 400 carrying a DPoP-Nonce header (consistent with the RS path). (#6) - Fix the DpopTokenRequestObservation docstring ("last write wins" → sticky-failure). (#7) Adds an expectedSuccessSlugs option to the client test helper and tightens the two nonce negative tests to pin the now-clean behavior (token-request-proof SUCCESS for no-as-nonce; only rs-nonce fails for no-rs-nonce). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * client/dpop: round-5 fixes — dedupe masking, wording, replay decision - dedupeSharedChecks was masking failures: it kept the last occurrence of a duplicate check ID regardless of status and ran in BOTH postures, so a FAILURE on a challenged POST (or an earlier attempt in a restarted flow) was silently dropped. Replace with an exported, unit-tested collapseDuplicateChecks that prefers the MOST-SEVERE occurrence, and only apply it in the nonce posture (the baseline keeps genuinely distinct repeated attempts). (R5 #1) - Reword the token-request-proof description: it can be SUCCESS for a client that presented a valid proof but never obtained a token (challenged, no retry), so it no longer claims "obtaining a DPoP-bound access token". (R5 #2) - Reword the as-nonce/rs-nonce failure messages to cover a client that DID retry but whose retry proof was rejected (not only "did not retry"). (R5 #3) - Document the deliberate replay decision: recording jti on challenged requests means re-sending the identical proof across the challenge/retry boundary trips replay detection — a genuine RFC 9449 §4.2 jti-uniqueness violation. (R5 #5) Adds src/scenarios/client/auth/dpop.test.ts unit-testing collapseDuplicateChecks (prefers failures, keeps INFO, preserves order) — the dedupe fix is now pinned. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * client/dpop: round-6 polish — pin collapse wiring; docstring - Add an integration assertion that the §8/§9 double-POST is collapsed to one token-request/pkce-* entry each in auth/dpop-nonce, so the requireNonce gating of collapseDuplicateChecks can't be silently deleted or inverted (previously mutation-survivable). runClientAgainstScenario now returns the emitted checks so callers can make occurrence-level assertions (backward-compatible). - Docstring: note the severity ladder ranks SKIPPED below SUCCESS, and that equal-severity ties keep the last occurrence. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Also add example of a client that passes the test.