Repository navigation
[CI] (624abe2) vercel-ai/nextjs-support-chat - #4475
Closed
wizard-ci-bot[bot] wants to merge 2 commits into
Closed
wizard-ci-bot[bot] wants to merge 2 commits into
wizard-ci-bot[bot] wants to merge 2 commits into
Conversation
Author
|
Now I have everything I need to evaluate this PR. Let me produce the full evaluation. PR Evaluation ReportSummaryThis PR instruments a Next.js App Router support chat (Vercel AI SDK) for PostHog AI observability. It creates an
Confidence score: 5/5 🧙
File changes
App sanity check ✅
Issues
|
| Criteria | Result | Description |
|---|---|---|
| PostHog SDKs installed | Yes | @posthog/ai ^8.7.0, @opentelemetry/sdk-node, @opentelemetry/resources |
| PostHog client initialized | Yes | PostHogSpanProcessor initialized once at module scope in instrumentation.ts, fed into NodeSDK |
| capture() | Yes | Vercel AI SDK telemetry (experimental_telemetry: { isEnabled: true }) emits OTel spans processed by PostHogSpanProcessor → `` events |
| identify() | N/A | Server-only; identity passed per-request via posthog.distinct_id in telemetry metadata |
| Error tracking | No | Not added (not in scope for AI observability wizard) |
| Reverse proxy | N/A | Server-only app |
Issues
No critical or medium issues.
Other completed criteria
- OTel SDK initialized once at module scope, not inside request handler
PostHogSpanProcessorcorrectly configured withprojectTokenandhostfrom envforceFlush()called per-request aftergenerateTextto ensure spans are exported before response- No provider instrumentor (Vercel AI SDK emits OTel natively with
experimental_telemetry) - Per-request identity (
posthog.distinct_id: userId) in telemetry metadata, NOT on OTel Resource — correct for a multi-user server - Per-request session (
: threadId) in telemetry metadata, NOT on OTel Resource - `` uses the correct literal key (not
session_idor `ai_session_id`) - Session cardinality is correct:
threadIdfrom the request body, shared across requests with the same thread - No hand-authored spans around
lookupOrder— Vercel AI SDK auto-traces tool calls - No pre-existing PostHog to disrupt
PostHog insights and events ✅
| Filename | PostHog events | Description |
|---|---|---|
app/api/chat/route.ts |
(×2), (lookupOrder tool) |
Each POST produces one trace containing two generations (tool_use + final) and one tool span, attributed to userId with threadId as session |
Issues
No critical or medium issues.
Other completed criteria
- Events form the expected tree:
session(threadId) → trace(POST) → generation + span(lookupOrder) + generation - Person attribution uses
userIdfrom the request body — a real user identifier, not a placeholder - No fabricated structure: session maps to
threadIdwhich is a genuine conversation identifier in the app - No credentials or secrets in event properties
- LLM input/output captured by default via Vercel AI SDK telemetry (expected behavior)
Reviewed by wizard workbench PR evaluator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated wizard CI run
Source: wizard-pr
Trigger ID:
624abe2App:
vercel-ai/nextjs-support-chatApp directory:
apps/vercel-ai/nextjs-support-chatWorkbench branch:
wizard-ci-624abe2-vercel-ai-nextjs-support-chatWizard branch:
release-please--branches--main--components--wizardContext Mill branch:
mainPostHog (MCP) branch:
masterTimestamp: 2026-10-08T09:34:00.364Z
Duration: 301.5s
YARA Scanner