feat(observability): add v2 genai tracing

[StarpTech]

Adds end-to-end V2 GenAI observability through OTLP. It records one trace per agent turn, model steps, HTTP and WebSocket transport, local tools, hosted-tool events, retries, compaction, subagents, and structured lifecycle failures. It also documents Dash0 setup and conversation navigation.

Only explicit operation boundaries create spans. Provider trace propagation remains disabled, URL queries are omitted, and prompts, model output, tool arguments, credentials, provider bodies, and raw exception messages are not recorded on spans. Existing retry, scheduling, and tool behavior remains unchanged.

Trace shape

invoke_agent build                       one agent turn
|-- event: session.input.promoted
|-- event: compaction/retry/hosted-tool lifecycle
|
|-- chat <model>                         model step 1
|   `-- POST / websocket.exchange        provider transport
|
|-- execute_tool <name>                  tool owned by the turn
|   `-- GET/POST                         HTTP owned by the tool
|
|-- execute_tool subagent
|   `-- invoke_agent explore             foreground child turn
|       |-- chat <child-model>
|       `-- execute_tool <child-tool>
|
`-- chat <model>                         continuation step
    `-- POST / websocket.exchange

Tool spans are siblings of model spans so concurrent tools can finish without outliving their model parent. Model transport spans require an explicit model span; unrelated Effect and HTTP spans remain disabled.

Conversations and subagents

Each turn is a separate root trace identified by gen_ai.conversation.id. A new turn links to the previous turn with opencode.link.type=previous_turn, allowing linked-trace navigation without keeping a conversation span open for hours. Turn links are bounded and process-local; gen_ai.conversation.id remains the canonical correlation key across restarts.

Foreground subagents remain nested beneath execute_tool subagent. Background subagents start separate root traces and link to the spawning tool with opencode.link.type=subagent. Parent and child Sessions are also correlated with opencode.session.parent.id and opencode.subagent.session.id.

Design principles

• Trace meaningful operations, not every Effect.
• Keep one bounded trace per agent turn and one chat <model> span per model step.
• Put transport and tool spans beneath the operation that owns them.
• Use span links for asynchronous or cross-turn relationships.
• Keep telemetry observational and fail-open.
• Record stable metadata, never sensitive content.
• Use structured logs for control-plane failures that do not need spans.

[github-actions]

This PR doesn't fully meet our contributing guidelines and PR template.

What needs to be fixed:

• PR description is missing required template sections. Please use the PR template.

Please edit this PR description to address the above within 2 hours, or it will be automatically closed.

If you believe this was flagged incorrectly, please let a maintainer know.

[github-actions]

The following comment was made by an LLM, it may be inaccurate:

Potential Duplicate Found

PR #34633: feat(observability): Implement OTel telemetry parity for agents and tools
#34633

Why it might be related:

• Both PRs focus on observability and tracing for agents and tools
• Both deal with instrumentation and telemetry in the GenAI stack
• PR feat(observability): Implement OTel telemetry parity for agents and tools #34633 implements "OTel telemetry parity" which overlaps with your PR's goal of semantic-convention spans and OTLP trace export
• Both address agent, model, transport, and tool activity tracking

Recommendation: Check if PR #34633 is still open/active and review its scope to understand if your v2 implementation is complementary or overlapping with existing work.

[jesse-schein]

This looks really awesome. I'm eager to get otel really wired up with opencode for a project I am working on. I can sort of hack opencode with the popular otel plugin to do what I want with modifications, but it would be great if opencode itself would accept traceparent header and stamp that on all the otel related traces like tools, sessions, etc.

I had AI write up some of my thoughts that I had it scan everywhere trying to see what was in the works for otel in v2, seems like otel issues/prs arent going anywhere in 1.X, likely in favor of v2 which makes a ton of sense.

One thing I'd love to see: could opencode accept a traceparent on the API (and ideally on the websocket path too) and use it as the parent for that turn?

What we're after

We run opencode serve long-lived in a sandbox and dispatch each turn over HTTP from our own service. Right now our spans and opencode's spans both make it to the backend fine, they just sit in separate traces, so you can't see the whole thing as one flamegraph. In Datadog or SigNoz it'd be really nice to click into a request and see the turn, the model calls and the tool calls hanging underneath whatever actually kicked it off.

OPENCODE_TRACEPARENT doesn't work for our setup because the server outlives any single request, so it's either one giant unbounded trace or nothing.

On the per-turn root thing

I saw each turn is deliberately its own root trace and I think that's the right default, a conversation span held open for hours would be worse. This wouldn't really change that. It'd still be one trace per turn, just rooted at the caller when a caller passes context.

Looks mostly wired already?

SessionTelemetry.TraceParent takes an AnySpan and Tracer.externalSpan() is exactly what you get from decoding a traceparent, so it seems like mostly a question of getting the header into that reference.

Also worth mentioning #37395 landed on v2 after you opened this and makes the server pick up inbound traceparent on request fibers, so you might end up in this code on rebase anyway.

Couple of things to consider

On websockets it'd have to ride on the message rather than the upgrade, since the connection isn't the turn.

And since message content can come from untrusted places it probably wants to be opt-in, so nobody can graft their spans onto someone else's trace.

Otherwise

If parenting isn't something you want to do here, even just a span link to the inbound context would help.

Happy to test against a real OTLP stack if that's useful.

[josephwangrb]

Would you consider supporting opt-in model content capture? The OTel GenAI conventions classify gen_ai.system_instructions, gen_ai.input.messages, and gen_ai.output.messages as opt-in, recommending that instrumentation keep them disabled by default but provide a way to enable them.

A flag such as OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, defaulting to false, could preserve the PR’s safe defaults while enabling debugging and evaluation use cases. Ideally, captured values would follow the OTel schemas and support filtering or truncation. Is this planned here or as a follow-up