Repository navigation
[lts-active] Keep requests through middleware in a single OpenTelemetry trace - #100226
Merged
sokra merged 1 commit intoOct 15, 2026
Merged
Conversation
…0136) ### What? Requests that pass through middleware / `proxy.ts` now produce **one OpenTelemetry trace**, instead of two or three. Before, with OTEL configured in `instrumentation.ts`: | Request | No outer span | With an outer span (APM, custom server) | |---|---|---| | Middleware continues to render (`next()` / rewrite) | 3 traces (2 before 16.3) | 2 traces (1 before 16.3) | | Middleware redirect / direct response | 2 traces (1 before 16.3) | 2 traces (1 before 16.3) | After: 1 trace in every case. ### Why? Two independent causes: 1. **Long-standing (13.5 → canary), #91282.** When middleware lets a request continue, the router server runs `BaseServer.handleRequest` twice: a middleware-invoke pass, then a render pass. Each pass extracts its context from the incoming request headers. Without an incoming `traceparent`, each pass started its own root trace. 2. **16.3 regression from #95306.** In-process Node.js middleware ran under `ROOT_CONTEXT` to make `Middleware.execute` a sibling of `handleRequest`. That also detached it from any outer span, so the middleware span always started an extra trace. Customers upgrading 16.2 → 16.3 see this as "doubled traces". ### How? When middleware lets a request continue, the render is always **nested inside the middleware span**, in every host: ``` next start / next dev: handleRequest GET (bubble, root) └─ middleware GET └─ GET /page separately-invoked middleware (Vercel, adapters, direct entrypoints), no parent: middleware GET (root) └─ GET /page host span / incoming traceparent P: P └─ middleware GET (next start: P → handleRequest GET → middleware GET) └─ GET /page ``` **Middleware → render.** The middleware adapter forwards the `Middleware.execute` span's own W3C context (`traceparent` / `tracestate`, via the configured propagator) as request-header overrides on `next()` and rewrite responses. Every host already applies these overrides to the downstream request, including the `next start` router server, so no new host protocol is needed. - Override lists replace the whole downstream header set. So: - **No user overrides:** the adapter lists every request header the middleware received, plus the trace headers. - **User overrides present:** they are kept, and the trace headers are added. - **User-set trace headers:** a `traceparent` / `tracestate` the middleware explicitly set (different from the one it received) wins. A value copied unchanged from the request, as with `new Headers(request.headers)`, is replaced so the render still nests. - Redirects and direct responses are untouched. **Request → middleware (self-hosted).** Once the middleware-invoke pass's `handleRequest` span is active, its W3C context is written onto the request headers. The middleware adapter (Node.js and the Edge sandbox) extracts it, so the middleware span is a child of that pass. The middleware-pass span keeps its existing name (`GET`); the route stays on the render-pass span. If there is no middleware span to forward (for example OTEL is registered only for Node.js while middleware runs on Edge), the render falls back to this context and still stays in one trace. **#95306 regression.** The `ROOT_CONTEXT` detachment is removed, so an outer span, an incoming `traceparent`, or the middleware-pass span now parents Node.js middleware normally. All of this only runs when OpenTelemetry is active and a valid span context exists. Without OTEL, no headers are added. One-time initialization traces stay separate by design: instrumentation registration, route-module prewarming, and the once-per-process lazy `RouteModule.prepare` of direct entrypoints. ### Tests - **New deploy-capable suite `test/e2e/app-dir/otel-middleware-single-trace`.** It runs Node.js `proxy.ts` and Edge `middleware.ts` with `@vercel/otel`. Middleware reports its active trace and span IDs; the render reports its active trace ID and the `traceparent` it received. The test asserts one trace and that the render's parent is the middleware span, so no collector is needed. It runs in dev, start and deploy mode, with no deploy gate. It covers: - continue to a page and to a route handler, plus rewrite, redirect and a direct response; - a request the middleware doesn't match, and an incoming `traceparent`; - user header overrides, including a deletion (the render still nests), and a user-set `traceparent`; - no internal `x-middleware-*` headers reaching the client. - **Existing collector suite `test/e2e/opentelemetry/instrumentation`.** The middleware tests now assert **real** trace IDs and the exact shape: the render span's parent is the middleware span, the middleware span's parent is the middleware-pass `handleRequest` (self-hosted) or the external / host parent. They also check every span exported for the request. - **Setups:** default and direct-entrypoint hosting; Edge and Node.js middleware; root context and incoming `traceparent`. - **Requests:** page, route handler, rewrite and redirect. - **New variant:** direct entrypoints with a host parent span. - **Unit tests** for the override helper (`forward-trace-context.test.ts`), including unchanged copied trace headers. ### Verification - The new suite's nesting checks and the collector shape checks fail before this change (render parented to `handleRequest` or the external span) and pass after it. - New suite passes in dev (Turbopack), start (Turbopack, webpack), and the same with `__NEXT_CACHE_COMPONENTS=true`. - Full `test/e2e/opentelemetry/instrumentation/` passes in start mode (Turbopack); the middleware tests also pass in dev (Turbopack) and start (webpack). - Passed on the previous revision (not re-run for the nesting change): `test/e2e/opentelemetry/client-trace-metadata`, `test/e2e/app-dir/otel-parent-span-propagation`, `test/e2e/middleware-request-header-overrides`, and the W3C trace-header fetch-cache test in `test/e2e/app-dir/app-static`. - Deploy: an earlier revision's trace-ID assertions passed on Vercel (default and Cache Components). The new parent-span assertion runs on Vercel for the first time in this revision's CI. Fixes #91282 <!-- NEXT_JS_LLM --> <!-- fleet ce0ecb23-b14c-4679-92a9-e676130f618e --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com> Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com> (cherry picked from commit 2e55993) Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
Contributor
Tests PassedCommit: 87e29dd |
github-actions
Bot
added
created-by: Turbopack team
sokra
marked this pull request as ready for review
October 15, 2026 08:35
mischnic
approved these changes
Oct 15, 2026
mischnic
pushed a commit
that referenced
this pull request
Oct 22, 2026
…0294, #100324, #100350, #100552, #100583) (#100542) Backports the OpenTelemetry request-trace stack to `releases/lts/active`: - #100349 Start the request span before route module preparation - #100294 Keep middleware and render in one OpenTelemetry trace as siblings (re-lands #100136, whose LTS backport #100226 was reverted in #100269) - #100324 docs: note that instrumentation-time spans don't join request traces - #100350 Emit instrumentation lifecycle spans in the request that registered it - #100552 Preserve deferred middleware instrumentation spans when the adapter throws - #100583 test: add non-adapter deploy snapshots to otel-middleware-trace. Without it, `otel-middleware-trace` fails the release deploy tests that deploy with `NEXT_ENABLE_ADAPTER=0`, as on v16.5.0-canary.23. The six canary merge commits are cherry-picked in order with `-x`, and all applied cleanly. The patch content is identical to canary's; only blob IDs and line offsets differ. ## Verification - `test/e2e/opentelemetry/instrumentation/`: `test-start-turbo` (122 passed) and `test-dev-turbo` (67 passed). The #100552 regression test fails without its `middleware.ts` change. - `test/e2e/app-dir/otel-middleware-trace`: `test-start-turbo`, `test-dev-turbo` and `test-start-webpack` (30 passed each, inline snapshots unchanged). The #100583 commit leaves the test file byte-identical to canary's. - Unit tests for `instrumentation-globals`, `tracer` and `forward-trace-context` (40 passed), plus `pnpm --filter=next types` - Built with `CI=true pnpm build-all` against the published `@next/swc` 16.4.2 binary. The backport touches no Rust. - PR CI deploys only through the adapter, so the non-adapter snapshots from #100583 are confirmed by the next release deploy run. <!-- NEXT_JS_LLM --> <!-- fleet ce0ecb23-b14c-4679-92a9-e676130f618e --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backports #100136 to
releases/lts/active.