Skip to content

[CI] (38ef7e6) fastapi/fastapi3-ai-saas - #2587

Closed
wizard-ci-bot[bot] wants to merge 1 commit into
mainfrom
wizard-ci-38ef7e6-fastapi-fastapi3-ai-saas
Closed

wizard-ci-bot[bot] wants to merge 1 commit into
mainfrom
wizard-ci-38ef7e6-fastapi-fastapi3-ai-saas

Conversation

@wizard-ci-bot

@wizard-ci-bot wizard-ci-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown

Automated wizard CI run

Source: wizard-pr
Trigger ID: 38ef7e6
App: fastapi/fastapi3-ai-saas
App directory: apps/fastapi/fastapi3-ai-saas
Workbench branch: wizard-ci-38ef7e6-fastapi-fastapi3-ai-saas
Wizard branch: release-please--branches--main--components--wizard
Context Mill branch: main
PostHog (MCP) branch: master
Timestamp: 2026-07-09T19:06:30.211Z
Duration: 474.0s

YARA Scanner

✓ 44 tool calls scanned, 0 violations detected

No violations: ✓ 44 clean scans

@wizard-ci-bot

wizard-ci-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown
Author

PR Evaluation Report

Summary

This PR integrates PostHog into a FastAPI AI SaaS application by adding the Python SDK, creating a custom ASGI middleware for user context identification, and instrumenting nine business-critical events across auth, content generation, API key management, and settings routes. However, a critical initialization mismatch means no events will actually be sent.

Files changed Lines added Lines removed
9 +177 -1

Confidence score: 4/5 👍

  • Instance vs. module-level client mismatch: The PR creates a Posthog() instance (posthog_client) in main.py but all route handlers and middleware use module-level functions (from posthog import capture, from posthog import new_context, identify_context). Module-level functions use a separate global client configured via posthog.api_key, which is never set. Per SDK docs, setup() returns a disabled client when api_key is blank, making all capture() calls no-ops — zero events will be sent. [CRITICAL]
  • .env.example not updated: POSTHOG_PROJECT_TOKEN and POSTHOG_HOST are not documented in .env.example, leaving collaborators unaware of required configuration. [MEDIUM]

File changes

Filename Score Description
app/config.py 4/5 Adds posthog_project_token and posthog_host to Pydantic Settings correctly
app/main.py 2/5 Creates Posthog() instance with enable_exception_autocapture=True and atexit shutdown, but the instance is never used by routes or middleware
app/middleware.py 3/5 Custom ASGI middleware wraps requests in PostHog context and identifies users from session cookie; uses module-level functions which won't connect to the initialized client
app/routers/auth.py 3/5 Instruments login, signup, logout with good event properties, but events are no-ops due to client mismatch
app/routers/generate.py 3/5 Instruments content generation success/failure with rich properties, same client issue
app/routers/api_keys.py 3/5 Instruments API key create/revoke, same client issue
app/routers/settings.py 3/5 Instruments settings update and password change, same client issue
requirements.txt 5/5 posthog>=3.0.0 added correctly
posthog-setup-report.md 4/5 Good documentation of events and next steps

App sanity check ⚠️

Criteria Result Description
App builds and runs Yes No syntax errors; all imports resolve to valid posthog package exports
Preserves existing env vars & configs Yes Existing app code, configs, and env vars are preserved
No syntax or type errors Yes All changed files have valid Python syntax
Correct imports/exports Yes Imports resolve correctly (posthog exports capture, new_context, identify_context, Posthog)
Minimal, focused changes Yes Changes are focused on PostHog integration only
Pre-existing issues None No pre-existing issues noted

Issues

  • .env.example missing PostHog variables: POSTHOG_PROJECT_TOKEN and POSTHOG_HOST are not documented in .env.example. Collaborators won't know to set these. Add them to .env.example. [MEDIUM]

Other completed criteria

  • Build configuration is valid — requirements.txt is parseable with correct package specifier
  • All changes are relevant to PostHog integration
  • Code follows existing codebase patterns (Pydantic Settings, router structure, middleware pattern)

PostHog implementation ❌

Criteria Result Description
PostHog SDKs installed Yes posthog>=3.0.0 added to requirements.txt
PostHog client initialized No Posthog() instance created in main.py but never used — all routes use module-level capture() which relies on unconfigured global client
capture() No 9 capture() calls exist but all use module-level function bound to unconfigured global client — events are silently dropped
identify() N/A Server-only app
Error tracking No enable_exception_autocapture=True is set on the instance, but middleware's new_context() uses module-level function not connected to that instance
Reverse proxy N/A Server-only app

Issues

  • Instance/module-level client mismatch — no events sent: main.py creates posthog_client = Posthog(settings.posthog_project_token, ...) but all route handlers import from posthog import capture (the module-level function). Module-level functions internally call setup(), which checks posthog.api_key — never set in this PR. Per SDK docs: "If api_key is missing or blank, the client is disabled and module-level calls become no-ops." Fix: Either (a) switch routes to use posthog_client.capture() / posthog_client.new_context() via dependency injection, or (b) replace the Posthog() constructor with module-level config (import posthog; posthog.api_key = ...; posthog.host = ...) and use module-level functions throughout. [CRITICAL]
  • Middleware new_context() also disconnected: The PostHogMiddleware uses from posthog import new_context, identify_context — module-level functions that won't use the posthog_client instance. Context identification and exception autocapture within the middleware are non-functional. [CRITICAL]

Other completed criteria

  • API key loaded from environment variable via Pydantic Settings (posthog_project_token)
  • Host correctly configured (https://us.i.posthog.com default, configurable via posthog_host)
  • atexit.register(posthog_client.shutdown) correctly registered for clean shutdown
  • posthog_client.flush() called in lifespan shutdown

PostHog insights and events ⚠️

Filename PostHog events Description
app/routers/auth.py user_signed_up, user_logged_in, user_logged_out Full auth lifecycle; signup/login include `` for person properties and method metadata
app/routers/generate.py content_generated, content_generation_failed AI content generation success and failure (insufficient credits) with rich properties
app/routers/api_keys.py api_key_created, api_key_revoked API key lifecycle tracking with active key count
app/routers/settings.py settings_updated, password_changed Account management events

Issues

  • Events are well-designed but non-functional: All 9 events have good names, meaningful properties, and enable funnels (signup → login → generate), but none will actually be sent due to the client initialization mismatch. [CRITICAL]

Other completed criteria

  • Events represent real user actions across the full product journey
  • Events enable product insights (signup-to-generation funnel, credit usage trends, auth patterns)
  • Events include relevant contextual properties (generation_type, credits_used, prompt_length, login_method)
  • No PII in event properties — user ID (integer) used for identification, not email
  • Event names are descriptive and consistent (snake_case, [noun]_[verb] pattern)

Reviewed by wizard workbench PR evaluator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants