Emit engine-independent MCP usage metadata in lockfiles by Copilot · Pull Request #54398 · github/gh-aw

Copilot · 2026-08-21T01:01:04Z

MCP adoption reporting depended on Claude-only # - mcp__... comment scraping, leaving non-Claude workflow engines undercounted. This adds structured MCP server/tool metadata to compiled lockfiles so analysis can work across engines.

Lockfile manifest
- Extends gh-aw-manifest with mcp_servers.
- Captures server names and tool allowlists from workflow configuration.
- Represents wildcard tool access as ["*"].
Compiler support
- Adds centralized MCP manifest collection for built-in, custom, safe-output, script, and enclave-backed MCP servers.
- Emits the manifest from the existing workflow header generation path.
Lockfile stats workflow
- Updates analysis guidance to prefer gh-aw-manifest.mcp_servers.
- Keeps legacy Claude comment parsing as a fallback for older lockfiles.
- Tracks fallback usage explicitly.

Example lockfile header metadata:

# gh-aw-manifest: {"engine":"copilot","mcp_servers":[{"name":"github","tools":["get_issue","list_issues"]},{"name":"custom-server","tools":["*"]}]}

Fixes [deep-report] Track MCP tool/server usage per-engine, not just via Claude comment-block scraping #53870

pr-sous-chef run: https://github.com/github/gh-aw/actions/runs/32477751698> Generated by 👨‍🍳 PR Sous Chef · gpt54 · 16.4 AIC · ⌖ 8.3 AIC · ⊞ 6.9K · ◷

_{Comment /souschef to run again}

github-actions · 2026-08-21T01:20:43Z

Hey @pelikhan 👋 — thanks for initiating work on MCP tool/server usage tracking per engine! This PR is a good start, but it's still in flight. Here's what's needed to move forward:

Complete the checklist — the PR body lists 5 tasks, all marked incomplete ([ ]). Once the compiler lockfile inspection and MCP manifest emission are done, mark them complete.
Add the actual diff — currently 0 files changed. The implementation should touch the compiler and/or MCP metadata emission layers.
Include tests — the checklist mentions "Add focused tests covering non-Claude/engine-agnostic manifest output" but no test files are in the diff yet. Add unit tests for the new engine-independent manifest logic once you've implemented the core feature.
Update PR description — once the checklist is done and the diff is ready, update the PR body to explain what changed, why, and any edge cases (e.g., handling workflows with no MCP servers).

Since this is a core-team agentic PR, your agent can continue implementing these steps. When ready for review, mark the draft as ready and the maintainers will take it from there.

If you need help breaking down the tasks or want to discuss the manifest format, feel free to sync up in the related issue #53870!

Generated by ✅ Contribution Check · auto · 54.9 AIC · ⌖ 5.14 AIC · ⊞ 9.2K · ◷

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2026-08-21T06:31:29Z

PR Triage

Category: feature
Risk: high
Priority score: 36/100 (impact 25 + urgency 0 + quality 11)
Recommended action: defer
CI: no checks reported
Draft PR — recommend undrafting before review.

Generated by 🔧 PR Triage Agent · auto · 79.3 AIC · ⌖ 6.15 AIC · ⊞ 8.3K · ◷

github-actions · 2026-08-21T10:18:18Z

✅ Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #54398

github-actions · 2026-08-21T10:18:19Z

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-08-21T10:18:20Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-08-21T10:18:20Z

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-08-21T10:18:21Z

❌ Design Decision Gate 🏗️ failed during design decision gate check.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-08-21T10:22:35Z

Comment Memory

reviewed_at: 2026-08-21T00:00:00Z
review_event: COMMENT
top_themes:
  - MCP manifest coverage looks consistent across compiler and generated lockfiles
  - No changed-line correctness regressions found in reviewed diff
files_reviewed:
  - pkg/workflow/mcp_manifest.go
  - pkg/workflow/compiler_yaml_header.go
  - pkg/workflow/safe_update_manifest.go
  - pkg/workflow/compiler_yaml_helpers_test.go
  - pkg/workflow/safe_update_manifest_test.go
  - .github/workflows/*.lock.yml
comment_count: 0

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 36 AIC · ⌖ 6.74 AIC · ⊞ 7K · ◷
_{Comment /review to run again}

github-actions

Verdict

No blocking issues found in the changed lines I reviewed.

Themes

The new `mcp_servers` manifest emission is wired through the shared header path, normalized deterministically, and covered by focused tests. I did not find a changed-line correctness, safety, or maintainability issue that justifies a review comment.

🔎 Code quality review by PR Code Quality Reviewer · gpt54 · 36 AIC · ⌖ 6.74 AIC · ⊞ 7K
_{Comment /review to run again}

github-actions

The lockfile changes correctly emit the new mcp_servers field in gh-aw-manifest headers. Spot-checking the visible diff:

mcp_servers is an array of {name, tools} objects ✅
Wildcard access is represented as ["*"] (e.g. agenticworkflows) ✅
Explicit tool allowlists are correctly scoped per server ✅
The field is placed after containers in the manifest, consistent across all lockfiles ✅

The source code changes (compiler, workflow analysis) are not visible in the capped diff, but the lockfile output is well-structured and matches the design described in the PR body. No blocking issues found.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · sonnet46 · 31.8 AIC · ⌖ 8.81 AIC · ⊞ 5.7K

github-actions

Ponytail review (over-engineering only).

net: -10 lines possible.

Generated by ✂️ Ponytail Reviewer for #54398 · auto · 32.9 AIC · ⌖ 8.66 AIC · ⊞ 7.3K
_{Comment /ponytail to run again}

github-actions · 2026-08-21T10:24:21Z

+		}
+	}
+	return keys
+}


L131-146: yagni: normalizedMapKeys uses reflect.ValueOf/Kind/MapKeys to extract keys, but all 3 call sites pass map[string]*T. A generic func normalizedMapKeys[V any](m map[string]V) []string does the same with zero reflection, matching sliceutil.SortedKeys[K,V] already used on L58 in this same file.

gh-aw-bot · 2026-08-21T19:01:59Z

@copilot

Please review the current PR state for any unresolved feedback or validation gaps, refresh the branch if GitHub offers it, and then run the pr-finisher skill.
Leave a concise maintainer handoff with validation status, merge readiness, and any remaining human action.

Run: https://github.com/github/gh-aw/actions/runs/32515777962

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 6.29 AIC · ⌖ 8.15 AIC · ⊞ 6.9K · ◷
_{Comment /souschef to run again}

…rack-mcp-usage # Conflicts: # .github/workflows/impeccable-skills-reviewer.lock.yml # .github/workflows/linter-miner.lock.yml Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-08-21T19:19:54Z

@copilot

Please review the current PR state for any unresolved feedback or validation gaps, refresh the branch if GitHub offers it, and then run the pr-finisher skill....

Maintainer handoff: refreshed main in c1bedad and regenerated lockfiles. MCP review feedback is already reflected in the current implementation. Workflow compilation and impacted pkg/workflow tests pass. Full validation is blocked by pre-existing copyloopvar failures in pkg/actionpins/actionpins_internal_test.go and completion tests failing with write |1: file already closed; CI runs on the prior head are action-required with no jobs/logs. Please re-trigger CI after this agent push and investigate/approve the external checks.

gh-aw-bot · 2026-08-21T19:23:51Z

@copilot

Please run the pr-finisher skill and leave a concise maintainer handoff for the current head. Before that, re-check the still-visible unresolved feedback and confirm whether each is already addressed on-branch or still needs code changes:

normalizedMapKeys reflection/YAGNI note: Emit engine-independent MCP usage metadata in lockfiles #54398 (comment)
no-mcp_servers / legacy-fallback ambiguity: Emit engine-independent MCP usage metadata in lockfiles #54398 (comment)
dynamic tool manifest over-reporting: Emit engine-independent MCP usage metadata in lockfiles #54398 (comment)
GitHub toolsets overcounting: Emit engine-independent MCP usage metadata in lockfiles #54398 (comment)
stringsFromAnySlice wildcard/unknown-shape handling: Emit engine-independent MCP usage metadata in lockfiles #54398 (comment)
gh-proxy manifest omission documentation/test: Emit engine-independent MCP usage metadata in lockfiles #54398 (comment)
safeoutputs tool ordering determinism: Emit engine-independent MCP usage metadata in lockfiles #54398 (comment)

Your latest handoff says the branch was refreshed and remaining validation blockers are mostly stale/action-required checks. If that is still true, say exactly which issues are already fixed on head c1bedad and which human follow-up is still needed.

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 46.5 AIC · ⌖ 9.8 AIC · ⊞ 9.3K · ◷
_{Comment /souschef to run again}

Copilot · 2026-08-21T19:38:48Z

@copilot

Please run the pr-finisher skill and leave a concise maintainer handoff for the current head. Before that, re-check the still-visible unresolved feedback and confirm whether each is alread...

Current head c1bedad already addresses all seven items: normalizedMapKeys is generic (no reflection); mcp_servers is always serialized including []; dynamic descriptors are excluded; GitHub toolsets/remote defaults are honored; unknown/string allowed shapes map to wildcard with tests; gh-proxy omission is documented and tested; and final manifest normalization sorts safe-output tools.

Maintainer handoff: make fmt, make test-unit, make test, and make recompile pass (the impacted-test merge-base probe warns because origin/main is absent, then the full unit suite passes). make lint remains blocked by two pre-existing copyloopvar findings in pkg/actionpins/actionpins_internal_test.go. Prior CI failures on de19f66 were inspected and are superseded by the MCP follow-up commits; CI on c1bedad is unverified because agent pushes do not trigger it. The PR is open/non-draft but GitHub reports blocked; unresolved review threads remain visible and need maintainer resolution/review. Human follow-up: resolve the already-addressed threads, clear/repair the unrelated lint findings, and re-trigger CI before merge.

gh-aw-bot · 2026-08-21T20:53:57Z

@copilot

This PR still shows merge conflicts (DIRTY). Please run make merge-main to resolve conflicts, then run the pr-finisher skill and leave a concise maintainer handoff for the refreshed head. If the currently visible unresolved threads are already addressed on-branch, say so explicitly with the commits/tests that cover them.

Run: https://github.com/github/gh-aw/actions/runs/32525213052

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 19.5 AIC · ⌖ 10.7 AIC · ⊞ 9.3K · ◷
_{Comment /souschef to run again}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-08-21T21:22:41Z

@copilot

This PR still shows merge conflicts (DIRTY). Please run make merge-main to resolve conflicts, then run the pr-finisher skill and leave a concise maintainer handoff for the refreshed he...

Resolved the DIRTY state by merging current origin/main in 76c2ed22c9 and regenerating all conflicted lockfiles from their Markdown sources. Followed up with 9ef10c5ed0 to correct documentation placement in the incoming workflow-builder split. make recompile, make fmt, targeted go test ./pkg/workflow -run TestWorkflowBuilder, and impacted pkg/workflow tests pass. make lint / final progress validation remain blocked by pre-existing repository-wide custom-linter findings; the full integration suite also hits a Cobra concurrent map writes panic in pkg/cli.

The visible MCP threads remain already addressed by 8940d96 and 87b252e: generic map keys, explicit empty mcp_servers, dynamic-tool exclusion, toolset filtering, wildcard handling, gh-proxy coverage, and deterministic output. Human follow-up: resolve those now-satisfied threads and re-trigger CI on the refreshed head.

pelikhan · 2026-08-21T21:31:15Z

@copilot resolve the merge conflicts in this pull request

gh-aw-bot · 2026-08-21T21:39:26Z

@copilot

This PR still shows merge conflicts (DIRTY) and the newest visible maintainer comment is an explicit merge-conflict request. Please run make merge-main on the current branch head, resolve any resulting generated lockfile conflicts by regenerating from the Markdown sources, then run the pr-finisher skill and leave a concise maintainer handoff.
If the currently visible unresolved MCP review threads are already addressed on-branch, say so explicitly with the exact commit(s) and validations that cover them.

Run: https://github.com/github/gh-aw/actions/runs/32529011926

Generated by 👨‍🍳 PR Sous Chef · gpt54 · 12.8 AIC · ⌖ 10.5 AIC · ⊞ 9.3K · ◷
_{Comment /souschef to run again}

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-08-21T21:47:12Z

@copilot resolve the merge conflicts in this pull request

Resolved in merge commit cd8d4cf. Regenerated all conflicted workflow lockfiles; make recompile, formatting, and impacted pkg/workflow tests pass. The final validation gate is blocked only by pre-existing custom-linter findings in unrelated workflow files.

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-08-21T22:06:10Z

@copilot

This PR still shows merge conflicts (DIRTY) and the newest visible maintainer comment is an explicit merge-conflict request. Please run make merge-main on the current branch head, reso...

Merged current origin/main in efb888d and regenerated all conflicted lockfiles from their Markdown sources. make recompile, make fmt, make test-unit, and make test pass. make lint and the final gate remain blocked by the existing copyloopvar findings in pkg/actionpins/actionpins_internal_test.go. The visible MCP threads are already addressed by 8940d96 and 87b252e; their manifest coverage remains validated by the passing workflow tests. CI on this agent-pushed head must be re-triggered by a maintainer.

…rack-mcp-usage

github-actions · 2026-08-22T04:53:39Z

🎉 This pull request is included in a new release.

Release: v0.87.4

Initial plan

b31a702

Copilot AI assigned Copilot and pelikhan Aug 21, 2026

Copilot started work on behalf of pelikhan August 21, 2026 01:01 View session

Copilot AI linked an issue Aug 21, 2026 that may be closed by this pull request

[deep-report] Track MCP tool/server usage per-engine, not just via Claude comment-block scraping #53870

Closed

github-actions Bot mentioned this pull request Aug 21, 2026

[Contribution Check Report] Contribution Check — 2026-08-21 #54403

Closed

feat: emit mcp usage manifest in lockfiles

de19f66

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add tracking for MCP tool/server usage per engine~~ Emit engine-independent MCP usage metadata in lockfiles Aug 21, 2026

Copilot finished work on behalf of pelikhan August 21, 2026 01:27

Copilot AI requested a review from pelikhan August 21, 2026 01:27

github-actions Bot added pr-action:defer pr-agent:copilot pr-priority:low pr-risk:high pr-type:feature labels Aug 21, 2026

github-actions Bot mentioned this pull request Aug 21, 2026

[PR Triage Report] PR Triage Report — 2026-08-21 (20 open Copilot PRs) #54454

Closed

pelikhan marked this pull request as ready for review August 21, 2026 10:17

Copilot AI balanced review requested due to automatic review settings August 21, 2026 10:17

Copilot started reviewing on behalf of pelikhan August 21, 2026 10:18 View session

github-actions Bot reviewed Aug 21, 2026

View reviewed changes

github-actions Bot approved these changes Aug 21, 2026

View reviewed changes

github-actions Bot reviewed Aug 21, 2026

View reviewed changes

github-actions Bot mentioned this pull request Aug 21, 2026

[PR Triage Report] PR Triage Report — 2026-08-21 (Run 32512501885) #54589

Closed

Copilot started work on behalf of gh-aw-bot August 21, 2026 19:02 View session

Merge remote-tracking branch 'origin/main' into copilot/deep-report-t…

c1bedad

…rack-mcp-usage # Conflicts: # .github/workflows/impeccable-skills-reviewer.lock.yml # .github/workflows/linter-miner.lock.yml Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot August 21, 2026 19:20

Copilot started work on behalf of gh-aw-bot August 21, 2026 19:24 View session

Copilot finished work on behalf of gh-aw-bot August 21, 2026 19:39

Copilot started work on behalf of gh-aw-bot August 21, 2026 20:54 View session

Copilot AI and others added 2 commits August 21, 2026 21:05

Merge remote-tracking branch 'origin/main'

76c2ed2

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Fix workflow builder documentation

9ef10c5

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot August 21, 2026 21:23

Copilot started work on behalf of pelikhan August 21, 2026 21:31 View session

Merge remote-tracking branch 'origin/main'

cd8d4cf

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan August 21, 2026 21:47

Copilot started work on behalf of gh-aw-bot August 21, 2026 21:47 View session

Merge remote-tracking branch 'origin/main'

efb888d

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot August 21, 2026 22:06

Merge remote-tracking branch 'origin/main' into copilot/deep-report-t…

98cb4dc

…rack-mcp-usage

pelikhan merged commit c149c09 into main Aug 21, 2026
3 checks passed

pelikhan deleted the copilot/deep-report-track-mcp-usage branch August 21, 2026 22:13

Uh oh!

Conversation

Copilot AI commented Aug 21, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 21, 2026

Uh oh!

github-actions Bot commented Aug 21, 2026

PR Triage

Uh oh!

github-actions Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 21, 2026

Comment Memory

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Verdict

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 21, 2026

Choose a reason for hiding this comment

Uh oh!

gh-aw-bot commented Aug 21, 2026

Uh oh!

Copilot AI commented Aug 21, 2026

Uh oh!

gh-aw-bot commented Aug 21, 2026

Uh oh!

Copilot AI commented Aug 21, 2026

Uh oh!

gh-aw-bot commented Aug 21, 2026

Uh oh!

Copilot AI commented Aug 21, 2026

Uh oh!

pelikhan commented Aug 21, 2026

Uh oh!

gh-aw-bot commented Aug 21, 2026

Uh oh!

Copilot AI commented Aug 21, 2026

Uh oh!

Copilot AI commented Aug 21, 2026

Uh oh!

Uh oh!

github-actions Bot commented Aug 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Aug 21, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Aug 21, 2026 •

edited

Loading

github-actions Bot commented Aug 21, 2026 •

edited

Loading

github-actions Bot commented Aug 21, 2026 •

edited

Loading

github-actions Bot commented Aug 21, 2026 •

edited

Loading

github-actions Bot commented Aug 21, 2026 •

edited

Loading