[azure-ai-projects] Add Agent Insights Samples by seangayler-msft · Pull Request #48889 · Azure/azure-sdk-for-python

seangayler-msft · 2026-09-03T19:16:48Z

Samples

Adds two self-contained samples under samples/agent_insights/. Each registers a temporary external agent, emits fictional traces, waits for ingestion, and deletes its own monitor and agent in finally.

On-demand: analyze the traces, read results, and resolve one insight.
Scheduled: enable a six-hour schedule and print its settings without waiting for scheduled analysis. Cleanup disables scheduling and cancels active runs before deletion.

Setup and cleanup use the shared agent_insights_util.py module. FOUNDRY_AGENT_NAME optionally sets the name prefix; it falls back to agent-insights-sample, with a unique suffix for each invocation. The project, connected Application Insights, analysis-model deployment, and permissions must already exist. The samples do not deploy infrastructure or change roles.

Supporting changes

Remove the separate recording fixture and optional resource templates. Keep focused Agent Insights tests under tests/agent_insights/.
Fix sync/async cancelled-run handling and preserve Foundry-Features on polling requests and the final GET.
Refresh both recordings to cover the complete sample lifecycle, including trace export and agent cleanup, with scoped redaction and offline playback.
Target package version 2.7.0.

Cancel VS Cancelled Patch Explanation

Cancellation is a core Agent Insights capability. The service returns cancelled, but Azure Core's default polling logic recognizes canceled instead. Without the adapter, polling can continue after a run is cancelled. The patch makes cancellation terminal for the polling engine while keeping the public poller.status() value as cancelled.

Preview-header propagation patch

The SDK adds Foundry-Features: AgentInsights=V1Preview to the initial request, but Azure Core's polling requests bypass that injection. The patch forwards the headers to both sync and async polling, including the final GET. Without it, polling can fail with HTTP 403 when the service enforces preview access. Samples do not need to add the header themselves.

azure-pipelines · 2026-09-03T19:17:10Z

Azure Pipelines: 11 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

🟢 Approval recommended

The reviewed changes have no unresolved issues blocking approval.

Pull request overview

Adds a synchronous Agent Insights sample demonstrating a safe monitor lifecycle.

Changes:

Demonstrates monitor creation, retrieval, listing, and cleanup.
Registers sample testing and LLM validation.
Documents the sample in the changelog.

File summaries

File	Description
`sdk/ai/azure-ai-projects/tests/samples/test_samples.py`	Registers the sample, pending an HTTP recording.
`sdk/ai/azure-ai-projects/tests/samples/llm_instructions.py`	Adds Agent Insights output validation mapping.
`sdk/ai/azure-ai-projects/samples/agent_insights/sample_agent_insights_basic.py`	Implements the monitor lifecycle sample.
`sdk/ai/azure-ai-projects/CHANGELOG.md`	Announces the new sample.

Review details

Files reviewed: 4/4 changed files
Comments generated: 0
Review effort level: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

azure-pipelines · 2026-09-03T20:44:50Z

Azure Pipelines: Successfully started running 1 pipeline(s). 10 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

🟡 Changes recommended

Live-test authentication and dependency ordering are broken, and fixture code is incompatible with supported Python 3.10.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 15/15 changed files
Comments generated: 4
Review effort level: Balanced

seangayler-msft · 2026-09-04T18:02:52Z

/azp run

azure-pipelines · 2026-09-04T18:02:58Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

seangayler-msft · 2026-09-04T18:03:44Z

/azp run

azure-pipelines · 2026-09-04T18:03:50Z

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

Copilot

🔵 Needs a closer look

It combines public polling behavior changes with live Azure resource provisioning and recorded integration workflows, warranting final human validation.

Review details

Files reviewed: 24/24 changed files
Comments generated: 0 new
Review effort level: Balanced

Use default final-response handling, preserve feature headers throughout polling, and record the simplified sample workflows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc

Copilot

🔵 Needs a closer look

Public LRO behavior, external Azure resources, and live recording artifacts warrant final human validation.

Review details

Files reviewed: 24/24 changed files
Comments generated: 0 new
Review effort level: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc

Copilot

🔵 Needs a closer look

Production polling behavior, recordings, and Azure resource provisioning warrant final human validation.

Review details

Files reviewed: 23/23 changed files
Comments generated: 0 new
Review effort level: Balanced

Remove duplicate error matrices and redundant mocked coverage while retaining polling headers, cancellation, cleanup races, future severity, and recording safeguards. Reuse the existing playback sleep fixture. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc

Copilot

🔵 Needs a closer look

The live Azure resource fixture and service-dependent recording workflow require final human validation.

Review details

Files reviewed: 23/23 changed files
Comments generated: 0 new
Review effort level: Balanced

Register temporary external agents, seed fictional telemetry, and clean up owned resources in each sample. Remove the separate provisioning fixture and keep focused lifecycle coverage. Recording refresh remains deferred. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc

Copilot

🟡 Changes recommended

The referenced recordings do not cover the new lifecycle, and recording sanitization would prevent deterministic playback.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 19/19 changed files
Comments generated: 3
Review effort level: Balanced

Record the complete temporary-agent sample lifecycle, scope telemetry redactions, and preserve trace relationships during playback. Align the package, changelog, sample requirements, and generated API metadata with version 2.7.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc

Copilot

🔵 Needs a closer look

The service-backed recordings and resource-creation cleanup paths warrant final human validation.

Review details

Files reviewed: 18/18 changed files
Comments generated: 0 new
Review effort level: Balanced

…ames and consolidate utility functions (#48970)

Copilot

🟡 Changes recommended

Sample helper imports are not isolated, causing focused and order-dependent test failures.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

sdk/ai/azure-ai-projects/tests/agent_insights/test_agent_insights_samples.py:38

This fixture has the same sibling-import failure: runpy.run_path() leaves samples/agent_insights off sys.path, while the loaded sample imports util. Add the sample directory before calling run_path() so the focused unit suite can collect this fixture independently.

    return runpy.run_path(str(path))

Files reviewed: 19/19 changed files
Comments generated: 3
Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.

Keep the merged shared helpers, allow an optional agent name prefix with the existing unique fallback, and remove the extra README. Give the helper module a distinct name to avoid collisions with other samples during test collection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc

Copilot

🟡 Changes recommended

Custom agent-name prefixes can produce service-invalid names and need validation or normalization.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 18/18 changed files
Comments generated: 1
Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.

Reject invalid or overlong prefixes before creating an agent and document the prefix constraints in both samples. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 965d2b3b-7439-4050-b224-cd487bcbb2bc

Copilot

🟢 Approval recommended

The sample lifecycle and polling fixes are consistent, documented, and comprehensively covered by focused tests.

Review details

Files reviewed: 18/18 changed files
Comments generated: 0 new
Review effort level: Balanced (auto)

Note

Copilot is running an experiment and ran this review at Balanced.

seangayler-msft self-assigned this Sep 3, 2026

github-actions Bot added the AI Projects label Sep 3, 2026

Base automatically changed from howie/shell to main September 3, 2026 19:44

seangayler-msft marked this pull request as ready for review September 3, 2026 20:00

Copilot AI balanced review requested due to automatic review settings September 3, 2026 20:00

seangayler-msft requested review from retry-recv (bobogogo1990), Darren Cohen (dargilco), Glenn Harper (glharper), Howie Leung (howieleung), kingernupur, Nikolay Rovinskiy (nick863) and Travis Angevine (trangevi) as code owners September 3, 2026 20:00

Copilot started reviewing on behalf of seangayler-msft September 3, 2026 20:01

View session

seangayler-msft force-pushed the sean/agent-insights-samples branch from a6a5773 to 2c62b9c Compare September 3, 2026 20:02

seangayler-msft marked this pull request as draft September 3, 2026 20:03

Copilot AI reviewed Sep 3, 2026

View reviewed changes

seangayler-msft marked this pull request as ready for review September 3, 2026 20:44

seangayler-msft force-pushed the sean/agent-insights-samples branch from 2c62b9c to 36680dd Compare September 4, 2026 17:18

Copilot AI review requested due to automatic review settings September 4, 2026 17:18

seangayler-msft marked this pull request as draft September 4, 2026 17:18

Copilot started reviewing on behalf of seangayler-msft September 4, 2026 17:18

View session

seangayler-msft changed the title ~~[azure-ai-projects] Add basic Agent Insights sample~~ [azure-ai-projects] Add reproducible Agent Insights sample Sep 4, 2026

Copilot AI reviewed Sep 4, 2026

View reviewed changes

seangayler-msft marked this pull request as ready for review September 4, 2026 18:05

Copilot started reviewing on behalf of seangayler-msft September 10, 2026 17:42

View session