feat: add Langfuse tracing integration (LangfuseBridge)#844
Open
hannguyen0712 wants to merge 5 commits into
Open
feat: add Langfuse tracing integration (LangfuseBridge)#844hannguyen0712 wants to merge 5 commits into
hannguyen0712 wants to merge 5 commits into
Conversation
Implements all features scoped in apache#206 via lifecycle hooks, built on OpenTelemetryBridge and the OTel-native Langfuse SDK: - One trace per application execution call (via the apache#203 interface) - One span per step; one span per __tracer span call - State observations: action inputs/read state and results/written state as observation input/output (capture_state=False to disable); log_attributes as observation metadata - Access to the Langfuse client via bridge.langfuse_client - app_id/partition_key map to Langfuse session/user (overridable) - Injects a should_export_span filter so Burr spans survive langfuse v4's default LLM-only export filtering (public: burr_span_export_filter) Uses the OTel bridge instead of the tracking/client.py pattern per the issue thread (May 31): the SDK's OTel rewrite and the closure of apache#205 made this the cleaner path, and it nests third-party OTel LLM instrumentation inside step spans automatically. Includes tests, docs, a runnable example, and a langfuse pyproject extra.
OTel attributes reject sequences of non-primitives, so list-of-dict values (e.g. chat_history logged by @trace) were dropped with a warning. Complex values are now JSON-serialized into observation metadata.
Route Burr spans through custom tracer providers and propagate session and user attributes to child instrumentation. Serialize mixed-type sequences before passing them to OpenTelemetry.
Propagate_attributes was added in langfuse 3.9.0 and Python 3.9 CI resolves an older 3.x (v4 requires py3.10+), so the unconditional import failed and was masked as 'plugin missing'. Falls back to per-span session/user attributes on older SDKs, with a regression test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a built-in Langfuse integration (#206): LangfuseBridge, a lifecycle hook that logs Burr application execution to Langfuse, built on OpenTelemetryBridge and the OTel-native Langfuse Python SDK.
Closes #206
Changes
burr/integrations/langfuse.pywithLangfuseBridge, attached via.with_hooks(LangfuseBridge()):__tracerspancapture_state=Falseto disable)__tracer.log_attributes()captured as observation metadata (complex values JSON-serialized, since OTel attributes only accept primitives)app_idmaps to the Langfuse session,partition_keyto the user (both overridable); the client is exposed asbridge.langfuse_clientshould_export_spanfilter (exported asburr_span_export_filterfor users constructing their own client), with a fallback for v3docs/reference/integrations/langfuse.rst), runnable example (examples/integrations/langfuse/), and alangfusepyproject extraNote on the divergence from the tracking-client pattern in the issue: the original issue proposed mirroring
burr/tracking/client.pyand depended on #205 (closed as not planned). Since then, the Langfuse SDK was rebuilt on OpenTelemetry and registers its span processor on the global tracer provider, so building on the existingOpenTelemetryBridgeavoids duplicating span-lifecycle machinery and nests third-party OTel LLM instrumentation (e.g.opentelemetry-instrumentation-openai) inside step spans for free. This direction was discussed and approved in the issue thread (May 31). Client access, which #205 would have provided via runtime inputs, is handled through constructor injection and the publicbridge.langfuse_clientattribute instead.Each execution call (
run,step,iterate, ...) becomes its own trace; repeated.step()calls therefore produce multiple traces, grouped under one Langfuse session viaapp_id(session_idis overridable if that's not desired). Async and streaming execution get the same coverage as the existing OTel bridge, since all execute methods flow through the same hooks.How I tested this
tests/integrations/test_burr_langfuse.py) run a real two-action app with a tracer span and logged attributes against an in-memory OTel exporter and a mocked Langfuse client, so CI needs no credentials. They assert the full span tree, parenting, session/user attributes, observation input/output payloads, metadata serialization,capture_state=False, constructor validation, the export filter, and error status.opentelemetry-instrumentation-openaiinstalled: screenshots below show the trace tree with the OpenAI generation (tokens/cost) nested inside the step spans, observation input/output, the session/user mapping, and error status on a failed run.tests/core,tests/visibility, and existing OTel integration tests pass; pre-commit clean.Verification screenshots (Langfuse Cloud)
Image 1: Langfuse trace list of one trace per execution call
Each
.run()call produces one Langfuse trace rooted at arunspan, with step and tracer spans nested beneath it.Image 2: Nested OpenAI generation view. Third-party OTel instrumentation composes automatically
With
opentelemetry-instrumentation-openaiinstalled, the LLM call appears as a Langfuse generation nested inside the_query_openaitracer span, with model, role-tagged prompt/completion messages, token usage (334 → 322, ∑ 656), and cost. No Langfuse-specific code in the action.Image 3: When a run is invalid, error status displays properly
A run with an invalid OpenAI key: the
chat_responsestep span and the generation are marked ERROR with the full 401 message, and Langfuse flags the trace via the child errors. The rootrunspan itself carries no error status, and this is the pre-existing core gap described in section Notes below (post_run_execute_callreceivesexception=None), not a limitation of this integration; the test comment marks the assertion to strengthen once that's fixed. Metadata also showsburr.sequence_idand theburr.integrations.langfusetracer scope.Image 4: Tracer spans + logged attributes as observation metadata
The
prepare_historyspan opened via__tracer(...)appears under its step, and__tracer.log_attributes(history_length=...)lands in the span's Langfuse metadata.Image 5: Step state captured as observation input/output
The
chat_responsestep's read state (prompt,chat_history) shows as observation input and its written state as observation output (disable withcapture_state=False).Image 6: Langfuse session view indicates that user mapping works correctly
Burr's
app_idmaps to the Langfuse session (both.run()calls of one application grouped together) andpartition_keyto the user (example-user), enabling Langfuse's session and user views.Notes
post_run_execute_callhooks always receiveexception=None; the wrappers in_call_execute_method_pre_post(burr/core/application.py) initializeexc = Noneinside atry/finallywith noexcept, so the exception is never captured. This affects the existingOpenTelemetryBridgetoo: root spans of failed runs are never marked errored (Langfuse still flags the trace via child-observation errors). This PR deliberately does not touch core; the error test asserts on the step span only, with a comment marking the assertion to strengthen once fixed. I have an 8-line fix with sync/async regression tests ready and am happy to file an issue and/or separate PR, whichever is preferred.Checklist