Skip to content

Start the request span before route module preparation - #100349

Merged
mischnic merged 5 commits into
canaryfrom
fleet/otel-request-span-before-prepare
Oct 21, 2026
Merged

Start the request span before route module preparation#100349
mischnic merged 5 commits into
canaryfrom
fleet/otel-request-span-before-prepare

Conversation

@sokra

@sokra sokra commented Oct 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Summary

Bottom of a stack: this PR → #100294 → #100324 → #100350 → #100392.

In direct entrypoints (Vercel, adapters, standalone handlers, i.e. without next-server), prepare route module ran before the request span started. Since 16.4 every request therefore produced an extra parentless prepare route module trace next to the request trace. Now it is part of the request span:

[P →] GET /page (BaseServer.handleRequest)
       └─ prepare route module → render …

How

  • New RouteModule.traceRequest (@internal, kept out of the published declaration types) starts the request span before prepare. It uses the caller's tracer. When the handler is wrapped by next-server, which already started the request span, it runs inside that active span instead.
  • Callers first await the new RouteModule.ensureInstrumentation (@internal), split out of prepareImpl: it registers instrumentation.ts, which usually installs the tracer provider, so a span started earlier would not be recorded. prepareImpl still calls it, and patches res.setHeader as before.
  • The app-page, app-route, pages and pages-api templates/handlers call ensureInstrumentation and then traceRequest, and handle the response through one code path: traceRequest passes the request span and the request's root span (the span that was active when the handler was invoked — next-server's request span or a platform/host span — or the request span itself), and the route name is propagated to the root span when it isn't the request span.
  • Parenting of the request span itself (incoming traceparent, host span, root) is unchanged.

Verification

  • Collector suite test/e2e/opentelemetry/instrumentation: direct entrypoint cold and warm requests for all four handler types, with and without an incoming traceparent and with a host span, assert RouteModule.prepare is inside the request's trace, and route names on the request and parent spans.
  • New deploy-capable e2e suite test/e2e/app-dir/otel-middleware-trace (moved down from Keep middleware and render in one OpenTelemetry trace as siblings #100294): proxy.ts and Edge middleware.ts variants, plus a custom server with a host span. The app records its spans with a SpanProcessor; the middleware and the render report the spans they can see, and the test asserts the stitched span tree (with http.target, http.route, next.route and http.status_code where set at report time) in inline snapshots for pages, route handlers, rewrites, redirects, direct responses, matcher misses, incoming traceparent, header overrides and a middleware-set traceparent, in dev, start, Cache Components and deploy. Here it records the current behavior (middleware and render in separate traces); Keep middleware and render in one OpenTelemetry trace as siblings #100294 shows the change.
  • Unit tests for the tracer changes (tracer.test.ts).

@github-actions

github-actions Bot commented Oct 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Tests Passed

Commit: bbac849

@sokra
sokra added this pull request to stack #100351 October 17, 2026 11:05
@sokra sokra mentioned this pull request Oct 17, 2026
Merged
@sokra sokra mentioned this pull request Oct 19, 2026
Merged
vercel-fleet-prod Bot and others added 3 commits October 19, 2026 15:20
@vercel-fleet-prod @mischnic
@sokra @lukesandberg
Keep route module preparation inside the request's trace for direct e…
b34eb5b 
…ntrypoints

Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
@vercel-fleet-prod @sokra
@lukesandberg @mischnic
Keep RouteModule.traceRequest out of published types
8eb8bd9 
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
@vercel-fleet-prod @sokra
@lukesandberg @mischnic
Move instrumentation registration out of RouteModule.traceRequest
2c34ecc 
Split `ensureInstrumentation` out of `prepareImpl` (renamed from
`prepareRuntime`). The build templates await it before calling
`traceRequest`, which no longer registers `instrumentation.ts` itself.
`res.setHeader` is patched in `prepareImpl` again.

Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
@mischnic
mischnic force-pushed the fleet/otel-request-span-before-prepare branch from 3047e21 to 2c34ecc Compare October 19, 2026 15:20
@mischnic
mischnic marked this pull request as ready for review October 20, 2026 13:14
vercel-fleet-prod Bot and others added 2 commits October 21, 2026 12:58
@vercel-fleet-prod @sokra
@lukesandberg @mischnic
Trace direct entrypoint responses through one span path
0a26c7c 
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
@vercel-fleet-prod @sokra
@lukesandberg @mischnic
Fill deployed otel-middleware-trace span tree snapshots
bbac849 
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
@sokra vercel-fleet-prod

sokra commented Oct 21, 2026

Copy link
Copy Markdown
Member Author

Reviewed bbac849

@mischnic
mischnic merged commit e3c2f63 into canary Oct 21, 2026
120 checks passed
@mischnic
mischnic deleted the fleet/otel-request-span-before-prepare branch October 21, 2026 18:30
mischnic added a commit that referenced this pull request Oct 21, 2026
@sokra @vercel-fleet-prod
@lukesandberg @mischnic
Keep middleware and render in one OpenTelemetry trace as siblings (#1…
4abb437 
…00294)

## Summary

Stacked on #100349 (request span before route module preparation).

Relands #100136 (reverted in f1a69d7) and changes the span shape so
middleware and the route are **siblings** in one trace per request.

### Resulting shapes

```
next start / next dev:
 handleRequest GET (bubble, root)
  ├─ middleware GET
  └─ GET /page

next start / next dev with host span / incoming traceparent P:
 P
  └─ handleRequest GET (bubble)
      ├─ middleware GET
      └─ GET /page

separately-invoked middleware (Vercel, adapters, direct entrypoints), no parent:
 GET (Next.request, root, inserted by the middleware adapter)
  ├─ middleware GET
  └─ GET /page

separately-invoked middleware with host span / incoming traceparent P:
 P
  ├─ middleware GET
  └─ GET /page
```

### How

- **Self-hosted:** next-server's middleware-invoke `handleRequest` pass
writes its W3C context onto the request (`traceparent`, and `tracestate`
only together with it: a received `tracestate` is removed when the
context has none). Middleware (Node in-process, or the Edge sandbox via
the header) and the render pass both continue it. The render pass forces
that extraction even under an active host span (custom server / APM), so
it nests under the bubble span; a different `traceparent` the middleware
explicitly set still wins.
- **Middleware adapter:** an active host span wins over a request
`traceparent`; otherwise the request's `traceparent` is continued, and
the middleware never changes it. Without any valid context (and OTEL
enabled) the adapter inserts a `Next.request` SERVER span (new
`NextSpan` enum) named after the method (`http.target` keeps the path).
When the downstream request doesn't already carry that shared parent,
its `traceparent`/`tracestate` are forwarded via the existing
`x-middleware-request-*` overrides, always as a pair: a user-set
`traceparent` keeps exactly the trace headers the user set. No headers
are added without OTEL.
- Request-header overrides replace the whole header set, so when the
user set no overrides, the received headers are listed alongside the
trace headers. There's no API to add a single header yet; that needs a
server and platform change.
- Node middleware is no longer detached via `ROOT_CONTEXT` (the 16.3
regression from #95306).

The inserted `Next.request` root ends when the middleware finishes, so
the render child (a separate invocation) can outlive it.

## Verification

- Trace shapes are asserted as printed span trees
(`test/lib/otel-span-tree.ts`, from #100349) in inline snapshots. Span
IDs are normalized away, but distinct traces keep distinct labels, so a
split trace shows up as `[trace A]` / `[trace B]`.
- `test/e2e/app-dir/otel-middleware-trace` (added in #100349 with the
previous behavior): this PR's snapshot diff shows middleware and render
becoming siblings in one trace, in dev, start, Cache Components and
deploy, for Node.js and Edge middleware and behind a custom server with
a host span (including an incoming `tracestate` not being paired with
the host span's context).
- Collector suite `test/e2e/opentelemetry/instrumentation`: the
middleware test asserts the full span tree in next start/dev, direct
entrypoints with/without host span, and the custom server (host span,
user-set `traceparent`).
- Unit tests for the tracer and `forwardTraceContextToRequest` changes.
- `test/e2e/middleware-request-header-overrides`,
`test/production/typescript-basic`,
`test/production/middleware-typescript` pass.

<!-- NEXT_JS_LLM -->


<!-- fleet ce0ecb23-b14c-4679-92a9-e676130f618e -->

---------

Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
@sokra sokra mentioned this pull request Oct 21, 2026
Merged
mischnic pushed a commit that referenced this pull request Oct 21, 2026
@vercel-fleet-prod @sokra
Start the request span before route module preparation (#100349)
1aae397 
## Summary

Bottom of a stack: this PR → #100294 → #100324 → #100350 → #100392.

In direct entrypoints (Vercel, adapters, standalone handlers, i.e.
without next-server), `prepare route module` ran before the request span
started. Since 16.4 every request therefore produced an extra parentless
`prepare route module` trace next to the request trace. Now it is part
of the request span:

```
[P →] GET /page (BaseServer.handleRequest)
       └─ prepare route module → render …
```

### How

- New `RouteModule.traceRequest` (`@internal`, kept out of the published
declaration types) starts the request span **before** `prepare`. It uses
the caller's tracer. When the handler is wrapped by next-server, which
already started the request span, it runs inside that active span
instead.
- Callers first await the new `RouteModule.ensureInstrumentation`
(`@internal`), split out of `prepareImpl`: it registers
`instrumentation.ts`, which usually installs the tracer provider, so a
span started earlier would not be recorded. `prepareImpl` still calls
it, and patches `res.setHeader` as before.
- The app-page, app-route, pages and pages-api templates/handlers call
`ensureInstrumentation` and then `traceRequest`, and handle the response
through one code path: `traceRequest` passes the request span and the
request's root span (the span that was active when the handler was
invoked — next-server's request span or a platform/host span — or the
request span itself), and the route name is propagated to the root span
when it isn't the request span.
- Parenting of the request span itself (incoming `traceparent`, host
span, root) is unchanged.

## Verification

- Collector suite `test/e2e/opentelemetry/instrumentation`: direct
entrypoint cold and warm requests for all four handler types, with and
without an incoming `traceparent` and with a host span, assert
`RouteModule.prepare` is inside the request's trace, and route names on
the request and parent spans.
- New deploy-capable e2e suite `test/e2e/app-dir/otel-middleware-trace`
(moved down from #100294): `proxy.ts` and Edge `middleware.ts` variants,
plus a custom server with a host span. The app records its spans with a
SpanProcessor; the middleware and the render report the spans they can
see, and the test asserts the stitched span tree (with `http.target`,
`http.route`, `next.route` and `http.status_code` where set at report
time) in inline snapshots for pages, route handlers, rewrites,
redirects, direct responses, matcher misses, incoming `traceparent`,
header overrides and a middleware-set `traceparent`, in dev, start,
Cache Components and deploy. Here it records the current behavior
(middleware and render in separate traces); #100294 shows the change.
- Unit tests for the tracer changes (`tracer.test.ts`).

<!-- NEXT_JS_LLM -->

<!-- fleet ce0ecb23-b14c-4679-92a9-e676130f618e -->

---------

Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
(cherry picked from commit e3c2f63)

Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
mischnic pushed a commit that referenced this pull request Oct 21, 2026
@vercel-fleet-prod @sokra
Keep middleware and render in one OpenTelemetry trace as siblings (#1…
25cac37 
…00294)

## Summary

Stacked on #100349 (request span before route module preparation).

Relands #100136 (reverted in f1a69d7) and changes the span shape so
middleware and the route are **siblings** in one trace per request.

### Resulting shapes

```
next start / next dev:
 handleRequest GET (bubble, root)
  ├─ middleware GET
  └─ GET /page

next start / next dev with host span / incoming traceparent P:
 P
  └─ handleRequest GET (bubble)
      ├─ middleware GET
      └─ GET /page

separately-invoked middleware (Vercel, adapters, direct entrypoints), no parent:
 GET (Next.request, root, inserted by the middleware adapter)
  ├─ middleware GET
  └─ GET /page

separately-invoked middleware with host span / incoming traceparent P:
 P
  ├─ middleware GET
  └─ GET /page
```

### How

- **Self-hosted:** next-server's middleware-invoke `handleRequest` pass
writes its W3C context onto the request (`traceparent`, and `tracestate`
only together with it: a received `tracestate` is removed when the
context has none). Middleware (Node in-process, or the Edge sandbox via
the header) and the render pass both continue it. The render pass forces
that extraction even under an active host span (custom server / APM), so
it nests under the bubble span; a different `traceparent` the middleware
explicitly set still wins.
- **Middleware adapter:** an active host span wins over a request
`traceparent`; otherwise the request's `traceparent` is continued, and
the middleware never changes it. Without any valid context (and OTEL
enabled) the adapter inserts a `Next.request` SERVER span (new
`NextSpan` enum) named after the method (`http.target` keeps the path).
When the downstream request doesn't already carry that shared parent,
its `traceparent`/`tracestate` are forwarded via the existing
`x-middleware-request-*` overrides, always as a pair: a user-set
`traceparent` keeps exactly the trace headers the user set. No headers
are added without OTEL.
- Request-header overrides replace the whole header set, so when the
user set no overrides, the received headers are listed alongside the
trace headers. There's no API to add a single header yet; that needs a
server and platform change.
- Node middleware is no longer detached via `ROOT_CONTEXT` (the 16.3
regression from #95306).

The inserted `Next.request` root ends when the middleware finishes, so
the render child (a separate invocation) can outlive it.

## Verification

- Trace shapes are asserted as printed span trees
(`test/lib/otel-span-tree.ts`, from #100349) in inline snapshots. Span
IDs are normalized away, but distinct traces keep distinct labels, so a
split trace shows up as `[trace A]` / `[trace B]`.
- `test/e2e/app-dir/otel-middleware-trace` (added in #100349 with the
previous behavior): this PR's snapshot diff shows middleware and render
becoming siblings in one trace, in dev, start, Cache Components and
deploy, for Node.js and Edge middleware and behind a custom server with
a host span (including an incoming `tracestate` not being paired with
the host span's context).
- Collector suite `test/e2e/opentelemetry/instrumentation`: the
middleware test asserts the full span tree in next start/dev, direct
entrypoints with/without host span, and the custom server (host span,
user-set `traceparent`).
- Unit tests for the tracer and `forwardTraceContextToRequest` changes.
- `test/e2e/middleware-request-header-overrides`,
`test/production/typescript-basic`,
`test/production/middleware-typescript` pass.

<!-- NEXT_JS_LLM -->

<!-- fleet ce0ecb23-b14c-4679-92a9-e676130f618e -->

---------

Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
(cherry picked from commit 4abb437)

Co-authored-by: Tobias Koppers <tobias.koppers@googlemail.com>
mischnic pushed a commit that referenced this pull request Oct 22, 2026
@sokra @vercel-fleet-prod @lukesandberg
[lts-active] Keep OpenTelemetry request traces together (#100349, #10…
37d603b 
…0294, #100324, #100350, #100552, #100583) (#100542)

Backports the OpenTelemetry request-trace stack to
`releases/lts/active`:

- #100349 Start the request span before route module preparation
- #100294 Keep middleware and render in one OpenTelemetry trace as
siblings (re-lands #100136, whose LTS backport #100226 was reverted in
#100269)
- #100324 docs: note that instrumentation-time spans don't join request
traces
- #100350 Emit instrumentation lifecycle spans in the request that
registered it
- #100552 Preserve deferred middleware instrumentation spans when the
adapter throws
- #100583 test: add non-adapter deploy snapshots to
otel-middleware-trace. Without it, `otel-middleware-trace` fails the
release deploy tests that deploy with `NEXT_ENABLE_ADAPTER=0`, as on
v16.5.0-canary.23.

The six canary merge commits are cherry-picked in order with `-x`, and
all applied cleanly. The patch content is identical to canary's; only
blob IDs and line offsets differ.

## Verification

- `test/e2e/opentelemetry/instrumentation/`: `test-start-turbo` (122
passed) and `test-dev-turbo` (67 passed). The #100552 regression test
fails without its `middleware.ts` change.
- `test/e2e/app-dir/otel-middleware-trace`: `test-start-turbo`,
`test-dev-turbo` and `test-start-webpack` (30 passed each, inline
snapshots unchanged). The #100583 commit leaves the test file
byte-identical to canary's.
- Unit tests for `instrumentation-globals`, `tracer` and
`forward-trace-context` (40 passed), plus `pnpm --filter=next types`
- Built with `CI=true pnpm build-all` against the published `@next/swc`
16.4.2 binary. The backport touches no Rust.
- PR CI deploys only through the adapter, so the non-adapter snapshots
from #100583 are confirmed by the next release deploy run.

<!-- NEXT_JS_LLM -->


<!-- fleet ce0ecb23-b14c-4679-92a9-e676130f618e -->

---------

Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>
@mikinovation mikinovation mentioned this pull request Oct 26, 2026
Merged
1 task
@github-actions github-actions Bot added the locked label Nov 5, 2026
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Nov 5, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@vercel-fleet-prod vercel-fleet-prod[bot] vercel-fleet-prod[bot] left review comments
@mischnic mischnic mischnic approved these changes

Assignees

No one assigned

Labels

created-by: Turbopack team PRs by the Turbopack team. locked tests type: next

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sokra @mischnic