Repository navigation
Description
TL;DR — Sentry and Eve both trace the same run, so all the AI numbers double. Only one of the two can be switched off, and it has to be Sentry's — which costs us automatic capture of tool errors.
Both emit a full gen_ai tree for the same run, so every model call is counted twice.
Eve's telemetry cannot be switched off. Eve turns it on when the app exports defineInstrumentation — there is no isEnabled toggle (eve 0.34.0, dist/src/public/instrumentation/index.d.ts:108-109: "The presence of a defineInstrumentation export implicitly enables telemetry"). That export is also the only place Eve gives an app to run Sentry.init (its setup hook), so the file cannot be removed without removing Sentry. The two paths are:
- Eve's spans: Eve's bundled
@ai-sdk/otelcallstrace.getTracer('gen_ai')on the global OTel API.Sentry.initregisters that global tracer provider (@sentry/nodesdk/initOtel.js), so these spans reach Sentry with no integration involved. - The SDK's spans: the default
VercelAIintegration subscribes to theai:telemetrydiagnostics channel that theaipackage publishes on every call, and starts a second set of spans.
Only path 2 can be switched off, so the integration is what has to go.
What filtering the integration costs
Exactly one signal: the subscriber's automatic captureException for failed tool executions.
const output = isObjectLike(result.output) ? result.output : undefined;
if (output?.type === "tool-error") {
captureToolError(span, data, output.error);
}@sentry/server-utils/build/cjs/vercel-ai/vercel-ai-dc-subscriber.js:266-270, capturing at :380-395 with mechanism: { type: "auto.vercelai.channel", handled: false } and vercel.ai.tool.name / vercel.ai.tool.callId tags.
That path is the only captureException in the subscriber. Model-call failures are not covered by it — see #11, which is a separate gap that filtering the integration does not cause and keeping it would not fix.
So the loss is scoped: tool errors stop opening issues on their own and have to be captured by the app. Everything else the integration would have reported is either still produced by Eve's telemetry or was never captured here in the first place.
Workaround: integrations: (defaults) => defaults.filter(i => i.name !== "VercelAI") in slack-agent-eve/agent/instrumentation.ts, plus an explicit capture in each tool.
Upstream: getsentry/sentry-javascript — the integration should stand down when an OTel gen_ai producer is already registered, so the two do not have to be traded against each other.
Status (2026-08-26)
- The workaround ships in the demo:
slack-agent-eve/agent/instrumentation.ts:214-215filtersVercelAI. - Verified live on
eve dev(eve 0.34.0,@sentry/node10.70.0): one turn produced exactly onegen_ai.invoke_agent, onegen_ai.agent_stepand onegen_ai.chatspan, with tokens counted once — https://sentry-developer-experience.sentry.io/explore/traces/trace/ab2cacff3bf0475e9b1c345fea7ed7e7 - Not fixed upstream as of
@sentry/node10.71.0 and11.0.0-alpha.1: the channel subscriber has no check for an already-registered gen_ai producer. - Because path 1 is plain OTel,
dataCollection.genAI(dataCollection.genAI is read only by Sentry AI integrations, so it silently does nothing for apps that emit gen_ai spans themselves #25) andtelemetry.isEnabled(telemetry: { isEnabled: true } overrides dataCollection.genAI and forces prompt/completion capture on #24) do not apply to Eve's spans; Eve'srecordInputs/recordOutputsdo.
Activity