⚡ Copilot Token Optimization2026-09-04 — Duplicate Code Detector #8091

Target Workflow: duplicate-code-detector

Source report: #7881
Estimated cost per run: $0.00 (not tracked in report, but token volume is highest of all workflows)
Total tokens per run: ~458,501 total / 4 runs ≈ but latest single measured run = 138,639 tokens
Cache hit rate: Not directly reported; working-set data shows a rebuild factor of 3.17x (cumulative input 137,910 vs peak input 43,516 tokens) across 4 invocations — indicating most of the context is being rebuilt on every turn rather than reused/cached.
LLM turns: 4 invocations (workflow caps at ≤4 turns)

Current Configuration

This workflow is already fairly well optimized (restricted GitHub toolset, pre-agent steps for deterministic work, threat-detection disabled, capped turns/issues). The main inefficiency is context rebuild overhead and prompt verbosity, not excessive tool surface.

Recommendations

1. Consolidate pre-computed analysis into a single file to cut redundant cat calls

Estimated savings: ~15–20K tokens/run (~12–15%)

Currently the prompt tells the agent to cat four separate files (code-metrics.txt, jscpd-top.json, grep-analysis.txt, existing-issues.json) across the session. Each additional bash tool round-trip re-sends the full running transcript (this is what's driving the 3.17x rebuild factor — 137,910 cumulative input tokens vs 43,516 peak). Merge these into one pre-agent step that writes a single /tmp/gh-aw/analysis-summary.md:

  - name: Consolidate analysis
    run: |
      {
        echo "## File Metrics"; cat /tmp/gh-aw/code-metrics.txt
        echo "## jscpd Top 15"; cat /tmp/gh-aw/jscpd-top.json
        echo "## Grep Patterns"; cat /tmp/gh-aw/grep-analysis.txt
        echo "## Existing Issues"; cat /tmp/gh-aw/existing-issues.json
      } > /tmp/gh-aw/analysis-summary.md

Then update the prompt's "Pre-Computed Analysis" section to reference only cat /tmp/gh-aw/analysis-summary.md (single read instead of four).

2. Trim prompt verbosity (200 lines → target ~120 lines)

Estimated savings: ~3–5K tokens/run (~3%)

The body repeats similar guidance in multiple places (Guidelines, Edge Cases, Phase 5/6, and the issue-body markdown template). Specific cuts:

• Merge "Guidelines" and "Edge Cases" sections — several bullets overlap with instructions already stated earlier (e.g., "No duplicates" repeats Phase 5 rules).
• Shorten the issue body markdown template by removing the verbose per-field explanations; keep just the structural skeleton (the agent can already infer field content from context).
• Remove the "Repository Context" bullet list of files — this is already implicitly covered by the pre-computed metrics/grep data.

3. Reduce jscpd/grep pre-step output size before it reaches the agent

Estimated savings: ~5–8K tokens/run (~5%)

find src -name '*.ts' ... | xargs wc -l | sort -rn | head -20 and the grep steps can produce noisy output (e.g., matches with long lines). Add cut -c1-200 or awk '{print substr($0,1,200)}' to grep-analysis to cap line length, and consider dropping head -20 down to head -10 for both metrics blocks since only the top few oversized files are actionable.

4. Lower the turn cap from 4 to 3 given the high rebuild factor

Estimated savings: ~25–30K tokens/run (~20%), proportional to reducing rebuilds

Since each additional turn re-sends nearly the full accumulated context (rebuild factor 3.17x), reducing Complete your analysis in ≤4 turns to ≤3 turns in the prompt would proportionally cut the highest-order cost. Combined with recommendation #1 (fewer separate cat calls), 3 turns should still be sufficient since all discovery data is pre-computed.

Expected Impact

Implementation Checklist

• Add a "Consolidate analysis" pre-agent step merging the 4 analysis files into /tmp/gh-aw/analysis-summary.md
• Update the "Pre-Computed Analysis" section of the prompt to reference the single consolidated file
• Trim the "Guidelines"/"Edge Cases" sections and shorten the issue-body template in the prompt
• Cap grep/metrics pre-step output line length and reduce head -20 to head -10 where appropriate
• Lower the turn cap from ≤4 turns to ≤3 turns in the prompt scope constraint
• Recompile: gh aw compile .github/workflows/duplicate-code-detector.md
• Post-process: npx tsx scripts/ci/postprocess-smoke-workflows.ts
• Verify CI passes on PR
• Compare token usage on new run vs baseline (138,639 tokens, 4 invocations, rebuild factor 3.17x)

Generated by Daily Copilot Token Optimization Advisor · copilot · auto · 36.4 AIC · ⊞ 10.7K · ◷