Skip to content

feat: add Langfuse tracing integration (LangfuseBridge)#844

Open
hannguyen0712 wants to merge 5 commits into
apache:mainfrom
hannguyen0712:feature/langfuse-integration
Open

feat: add Langfuse tracing integration (LangfuseBridge)#844
hannguyen0712 wants to merge 5 commits into
apache:mainfrom
hannguyen0712:feature/langfuse-integration

Conversation

@hannguyen0712

@hannguyen0712 hannguyen0712 commented Jul 22, 2026

Copy link
Copy Markdown

Adds a built-in Langfuse integration (#206): LangfuseBridge, a lifecycle hook that logs Burr application execution to Langfuse, built on OpenTelemetryBridge and the OTel-native Langfuse Python SDK.

Closes #206

Changes

  1. New burr/integrations/langfuse.py with LangfuseBridge, attached via .with_hooks(LangfuseBridge()):
  • One trace per application execution call, one span per step, one span per __tracer span
  • Action inputs/read state and results/written state captured as observation input/output (capture_state=False to disable)
  • __tracer.log_attributes() captured as observation metadata (complex values JSON-serialized, since OTel attributes only accept primitives)
  • app_id maps to the Langfuse session, partition_key to the user (both overridable); the client is exposed as bridge.langfuse_client
  • Langfuse SDK v4 only exports LLM-relevant spans by default, so the bridge injects a should_export_span filter (exported as burr_span_export_filter for users constructing their own client), with a fallback for v3
  1. Docs page (docs/reference/integrations/langfuse.rst), runnable example (examples/integrations/langfuse/), and a langfuse pyproject extra

Note on the divergence from the tracking-client pattern in the issue: the original issue proposed mirroring burr/tracking/client.py and depended on #205 (closed as not planned). Since then, the Langfuse SDK was rebuilt on OpenTelemetry and registers its span processor on the global tracer provider, so building on the existing OpenTelemetryBridge avoids duplicating span-lifecycle machinery and nests third-party OTel LLM instrumentation (e.g. opentelemetry-instrumentation-openai) inside step spans for free. This direction was discussed and approved in the issue thread (May 31). Client access, which #205 would have provided via runtime inputs, is handled through constructor injection and the public bridge.langfuse_client attribute instead.

Each execution call (run, step, iterate, ...) becomes its own trace; repeated .step() calls therefore produce multiple traces, grouped under one Langfuse session via app_id (session_id is overridable if that's not desired). Async and streaming execution get the same coverage as the existing OTel bridge, since all execute methods flow through the same hooks.

How I tested this

  • 6 integration tests (tests/integrations/test_burr_langfuse.py) run a real two-action app with a tracer span and logged attributes against an in-memory OTel exporter and a mocked Langfuse client, so CI needs no credentials. They assert the full span tree, parenting, session/user attributes, observation input/output payloads, metadata serialization, capture_state=False, constructor validation, the export filter, and error status.
  • Verified end to end against Langfuse Cloud with opentelemetry-instrumentation-openai installed: screenshots below show the trace tree with the OpenAI generation (tokens/cost) nested inside the step spans, observation input/output, the session/user mapping, and error status on a failed run.
  • Full tests/core, tests/visibility, and existing OTel integration tests pass; pre-commit clean.

Verification screenshots (Langfuse Cloud)

Image 1: Langfuse trace list of one trace per execution call

langfuse-trace-list

Each .run() call produces one Langfuse trace rooted at a run span, with step and tracer spans nested beneath it.

Image 2: Nested OpenAI generation view. Third-party OTel instrumentation composes automatically

langfuse-openai-query-token-view

With opentelemetry-instrumentation-openai installed, the LLM call appears as a Langfuse generation nested inside the _query_openai tracer span, with model, role-tagged prompt/completion messages, token usage (334 → 322, ∑ 656), and cost. No Langfuse-specific code in the action.

Image 3: When a run is invalid, error status displays properly

A run with an invalid OpenAI key: the chat_response step span and the generation are marked ERROR with the full 401 message, and Langfuse flags the trace via the child errors. The root run span itself carries no error status, and this is the pre-existing core gap described in section Notes below (post_run_execute_call receives exception=None), not a limitation of this integration; the test comment marks the assertion to strengthen once that's fixed. Metadata also shows burr.sequence_id and the burr.integrations.langfuse tracer scope.

Image 4: Tracer spans + logged attributes as observation metadata

The prepare_history span opened via __tracer(...) appears under its step, and __tracer.log_attributes(history_length=...) lands in the span's Langfuse metadata.

Image 5: Step state captured as observation input/output

langfuse-chat-response-input-output

The chat_response step's read state (prompt, chat_history) shows as observation input and its written state as observation output (disable with capture_state=False).

Image 6: Langfuse session view indicates that user mapping works correctly

Burr's app_id maps to the Langfuse session (both .run() calls of one application grouped together) and partition_key to the user (example-user), enabling Langfuse's session and user views.

Notes

  • Pre-existing core gap found while testing: post_run_execute_call hooks always receive exception=None; the wrappers in _call_execute_method_pre_post (burr/core/application.py) initialize exc = None inside a try/finally with no except, so the exception is never captured. This affects the existing OpenTelemetryBridge too: root spans of failed runs are never marked errored (Langfuse still flags the trace via child-observation errors). This PR deliberately does not touch core; the error test asserts on the step span only, with a comment marking the assertion to strengthen once fixed. I have an 8-line fix with sync/async regression tests ready and am happy to file an issue and/or separate PR, whichever is preferred.
  • The original issue said to open a follow-up for approach (2); I can file a tracking issue for the deferred items (deeper model reconciliation per Add metadata capture #46, Langfuse-native generation types, scoring helpers) if useful.

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

Implements all features scoped in apache#206 via lifecycle hooks, built on
OpenTelemetryBridge and the OTel-native Langfuse SDK:

- One trace per application execution call (via the apache#203 interface)
- One span per step; one span per __tracer span call
- State observations: action inputs/read state and results/written state
  as observation input/output (capture_state=False to disable);
  log_attributes as observation metadata
- Access to the Langfuse client via bridge.langfuse_client
- app_id/partition_key map to Langfuse session/user (overridable)
- Injects a should_export_span filter so Burr spans survive langfuse
  v4's default LLM-only export filtering (public: burr_span_export_filter)

Uses the OTel bridge instead of the tracking/client.py pattern per the
issue thread (May 31): the SDK's OTel rewrite and the closure of apache#205
made this the cleaner path, and it nests third-party OTel LLM
instrumentation inside step spans automatically.

Includes tests, docs, a runnable example, and a langfuse pyproject extra.
OTel attributes reject sequences of non-primitives, so list-of-dict values
(e.g. chat_history logged by @trace) were dropped with a warning. Complex
values are now JSON-serialized into observation metadata.
@github-actions github-actions Bot added area/integrations External integrations (LLMs, frameworks) area/website burr.apache.org website area/ci Workflows, build, release scripts area/examples Relates to /examples labels Jul 22, 2026
@hannguyen0712 hannguyen0712 mentioned this pull request Jul 22, 2026
5 tasks
@hannguyen0712 hannguyen0712 changed the title Feature: add Langfuse tracing integration (LangfuseBridge) feat: add Langfuse tracing integration (LangfuseBridge) Jul 22, 2026
jernejfrank and others added 3 commits July 23, 2026 15:53
Route Burr spans through custom tracer providers and propagate session and user attributes to child instrumentation. Serialize mixed-type sequences before passing them to OpenTelemetry.
Propagate_attributes was added in langfuse 3.9.0 and  Python 3.9 CI resolves an older 3.x (v4 requires py3.10+), so the unconditional import failed and was masked as 'plugin missing'. Falls back to per-span session/user attributes on older SDKs, with a regression test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/ci Workflows, build, release scripts area/examples Relates to /examples area/integrations External integrations (LLMs, frameworks) area/website burr.apache.org website

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Langfuse integration

2 participants