Skip to content

[CI] (69eeefc) manual-capture/node-http-chat - #4075

Closed
wizard-ci-bot[bot] wants to merge 1 commit into
mainfrom
wizard-ci-69eeefc-manual-capture-node-http-chat
Closed

wizard-ci-bot[bot] wants to merge 1 commit into
mainfrom
wizard-ci-69eeefc-manual-capture-node-http-chat

Conversation

@wizard-ci-bot

@wizard-ci-bot wizard-ci-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown

Automated wizard CI run

Source: wizard-pr
Trigger ID: 69eeefc
App: manual-capture/node-http-chat
App directory: apps/manual-capture/node-http-chat
Workbench branch: wizard-ci-69eeefc-manual-capture-node-http-chat
Wizard branch: posthog/functional-a1-min
Context Mill branch: main
PostHog (MCP) branch: master
Timestamp: 2026-09-22T15:54:28.798Z
Duration: 163.6s

YARA Scanner

✓ 31 tool calls scanned, 0 violations detected

No violations: ✓ 31 clean scans

@wizard-ci-bot

wizard-ci-bot Bot commented Sep 22, 2026

Copy link
Copy Markdown
Author

PR Evaluation Report

Summary

This PR adds manual and capture to a plain-fetch chat app that has no vendor LLM SDK. It reuses the existing PostHog client from src/posthog.ts, adds an AiContext type to thread trace/session/user identity through calls, and captures events with correct property keys. No existing code was altered beyond threading the new context parameter.

Files changed Lines added Lines removed
4 +107 -4

Confidence score: 5/5 🧙

  • No issues of CRITICAL or MEDIUM severity found. The implementation matches the README's expected tree precisely.

File changes

Filename Score Description
src/index.ts 5/5 Added AiContext type, capture inside `complete()`, capture for tool dispatch, shared traceId per ask(), sessionId from threadId
.gitignore 5/5 Ignores .env to prevent credential leakage
.posthog-wizard-cache/.posthog-ai.json 5/5 Standard wizard cache metadata
posthog-ai-observability-report.md 5/5 Accurate description of changes and expected trace hierarchy

App sanity check ✅

Criteria Result Description
App builds and runs Yes Report confirms npm run build (tsc --noEmit) succeeded
Preserves existing env vars & configs Yes No new env vars; existing POSTHOG_PROJECT_API_KEY and POSTHOG_HOST from posthog.ts unchanged
No syntax or type errors Yes Valid TypeScript; randomUUID, performance.now(), URL are all available in Node
Correct imports/exports Yes randomUUID from node:crypto is correct; existing posthog and lookupOrder imports preserved
Minimal, focused changes Yes Only AI observability capture added; tool loop and message construction unchanged
Pre-existing issues None —

Issues

None.

Other completed criteria

  • Existing PostHog client from src/posthog.ts reused — no second instance created
  • posthog.shutdown() already present in main() and error handler — flush before exit preserved
  • No new dependencies added; posthog-node ^4.2.0 already existed
  • identify() and chat_message_sent capture untouched
  • .env gitignored; no credentials in source

PostHog implementation ✅

Criteria Result Description
PostHog SDKs installed Yes Existing posthog-node ^4.2.0 reused; no additional packages needed
PostHog client initialized Yes Singleton from src/posthog.ts reused — not duplicated
capture() Yes Manual and captures with correct event names and property keys
identify() N/A Pre-existing identify() call untouched; server-side script
Error tracking No No error tracking added (not in scope for AI observability wizard)
Reverse proxy N/A Server-side Node script

Issues

None.

Other completed criteria

  • Instrumentation is set up at call sites (manual capture path — no module-scope init needed)
  • Every includes:, , , , , , , , , ``
  • Shared traceId per ask() ensures all calls in one request land in one trace
  • uses the literal key (not a variant spelling)
  • Session id is Thread.threadId ('thread_abc'), shared across turns — correct cardinality
  • `` for lookup_order is a real tool execution, not fabricated structure
  • `` set on every generation and span, pointing to the trace id

PostHog insights and events ✅

Filename PostHog events Description
src/index.ts `` Captured for every complete() call with full model metadata, I/O, tokens, and latency
src/index.ts `` Captured for lookup_order tool execution with input/output state and latency
src/index.ts chat_message_sent (pre-existing) Product analytics event, untouched

Issues

None.

Other completed criteria

  • First ask() emits: generation (tool-selection) → span (lookup_order) → generation (final answer) — 3 events on one trace
  • Second ask() emits: generation — 1 event on a separate trace
  • Both traces share = 'thread_abc'
  • All events attributed to user_123 (the thread's stable user ID)
  • No PII or secrets in event properties; LLM I/O is expected content

Reviewed by wizard workbench PR evaluator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants