Repository navigation
Conversation
…event A failed model call set error.type on its span and on the duration metric, but not on the gen_ai.client.inference.operation.details event, so the event of a failed call had no error type. The completion event is now emitted from an exit callback that receives the exception, and carries the same error.type as the span.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
2. Or, if no issue exists, describe the change:
Problem:
When a model call fails, ADK sets
error.typeon the inference span and on theduration metric, but not on the
gen_ai.client.inference.operation.detailsevent. The event of a failed call therefore has no error type, and the
instrumentor's event for the same call has
error.type(429for aresource-exhausted error). The divergence is recorded as
adk_bugintests/unittests/telemetry/functional_divergences.json.Solution:
The completion event is emitted from an exit callback on the inference span's
exit stack. That callback was registered with
ExitStack.callback, which dropsthe exception that is unwinding through it. It is now registered with
push,which passes the exception, and when the call failed the event carries
error.typefromtracing.resolve_error_type, the same value as the span.The span and the duration metric are unchanged.
Testing Plan
Unit Tests:
The change is covered by the functional telemetry goldens. Three inference-error
goldens gain the
error.typeline on the completion event, and theadk_bugdivergence entry is removed:
functional_goldens/agent/inference-error-resource-exhausted-schema-v2.json(
error.type: "429")functional_goldens/agent/inference-error-valueerror-schema-v2.json(
error.type: "ValueError")functional_goldens/agent/inference-error-valueerror-span-and-event-schema-v2.json(
error.type: "ValueError")Run with the goldens regenerated (
python -m tests.unittests.telemetry.regeneratewas used to record them), the
tests/unittests/telemetrysuite was run bothbefore and after the change in the same environment:
101 failed, 885 passed, 3 xfailedin both runs, with the same set of failing tests. The failures arethe telemetry suite's divergence checks against the locally installed
opentelemetry-instrumentation-google-genaifrom its development branch, whichthe committed divergence file does not describe. They are not caused by this
change.
test_google_cloud.pywas excluded because it imports a symbol thelocally installed
opentelemetry-apidoes not provide.Manual End-to-End (E2E) Tests:
Not run. The failing-call path is covered by the functional goldens above.
Checklist