Skip to content

Add Visual Studio 2026 (GitHub Copilot in VS) as a harness #283

Description

Problem

AI Engineer Coach reads sessions from VS Code (Copilot Chat), GitHub Copilot App/CLI (~/.copilot/session-state), Xcode, and the external harnesses Claude Code, Codex CLI and OpenCode (src/core/parser-harnesses.ts). There is no collector for Visual Studio 2026 (18.x).

Developers using GitHub Copilot Chat / agent mode inside Visual Studio (.NET, C++ shops) therefore get no Timeline, Output, Patterns or Anti-Patterns coverage for that work — even though the extension (or the Copilot app canvas) runs on the same Windows machine and the session data already exists locally. Since the April 2026 update, Visual Studio keeps a dedicated chat history panel (titles, message previews, timestamps), so the sessions are persisted and parseable.

Related: #151 (additional harnesses on hold). Xcode support shows the project already accepts non-VS Code IDEs as first-class harnesses, which is the precedent for this request.

Proposed Solution

  1. New collector Visual Studio in src/core/parser-harnesses.ts, backed by src/core/parser-visualstudio.ts following the parser-claude / parser-codex / parser-opencode pattern: findVisualStudioDirs() + parseVisualStudioSessions(dir, editLocIndex). Add 'VisualStudio' to EXTERNAL_HARNESS_SET.
  2. Discovery, Windows only (process.platform === 'win32'):
    • primary: the VS 2026 persisted chat-session store (undocumented; expected under %LOCALAPPDATA%\Microsoft\VisualStudio\18.0_<instance>\ — path and schema to confirm, I can provide anonymised samples);
    • fallback: %LOCALAPPDATA%\Temp\**\*VSGitHubCopilot.chat.log — the format VS 2022 used (system prompt, analysed context, user/assistant turns). See https://github.com/orgs/community/discussions/57190#discussioncomment-11536706.
  3. Mapping to Session:
    • workspace = solution (.sln / .slnx) or project folder found in the log context;
    • user / assistant turns → requests/responses with timestamps;
    • model id → model usage table;
    • agent-mode tool calls (e.g. get_symbol_call_hierarchy, Debugger agent) → tool-mastery metrics;
    • generated code blocks → EditLocIndex (code output by language).
  4. Harness label Visual Studio in the sidebar/harness breakdown, distinct from VS Code.
  5. hasExternalHarnessSources() returns true when VS logs exist, so the dashboard opens on a machine with no VS Code workspace storage (same reasoning as fix: load dashboard when only non-VS Code harness logs exist (Claude/Codex/OpenCode) #53).
  6. Fail soft: skip unparsable files and surface a count through the harness progress handlers, since the VS log format is undocumented and may change between updates.

I'm happy to contribute this as a PR (parser + sample logs + tests) rather than ask the maintainers to build it.

Alternatives Considered

  • Use Copilot CLI from the Visual Studio terminal — works today and lands in the GitHub Copilot CLI harness, but misses the IDE chat/agent sessions, which is where most VS users work.
  • Export each VS chat manually (copy as Markdown) and convert to the VS Code chatSessions JSON format — no stable export format from VS, and not sustainable day to day.
  • Keep a separate fork with the parser — fragments the "any harness, one dashboard" promise and loses the shared rule set.
  • Wait for Microsoft to document the VS session store — no public timeline; a tolerant parser with a Temp-log fallback is useful now.

Activity

  1. github-actions commented on Oct 7, 2026

    @github-actions

    🎯 Triage report

    This is a well-scoped feature request to add Visual Studio 2026 Copilot chat/agent sessions as a new harness source, with concrete parser touchpoints already identified. The request is actionable for planning, but implementation still depends on validating the actual Windows session-store location/schema and getting representative sample logs because the format is undocumented.

    📊 Assessment

    Dimension Value Reasoning
    Type Feature The issue requests new platform/harness support rather than describing a regression in an existing supported harness.
    Labels enhancement Existing label already matches; I did not add more because platform-specific or backlog labeling would be speculative from the issue alone.
    Severity Low This is a missing capability for a subset of users rather than a documented break in a currently supported workflow.
    Coding agent Needs more info The code touchpoints are clear, but a safe implementation still needs confirmed sample logs / storage schema for Visual Studio 2026.

    🔗 Similar issues

    💡 Notes and suggestions

    Useful implementation/decomposition notes for maintainers:

    • Validate inputs first: confirm the real VS 2026 persisted session path/layout and capture anonymized sample logs before committing to a parser shape.
    • Stage the work: an incremental first milestone could support the Temp-log fallback path only, then add the persisted-session store once the schema is understood.
    • Likely code areas: src/core/parser-harnesses.ts, a new src/core/parser-visualstudio.ts, harness labeling, and the external-harness gate precedent referenced from fix: load dashboard when only non-VS Code harness logs exist (Claude/Codex/OpenCode) #53.
    • Parsing concerns: keep Windows-only discovery, tolerate undocumented schema drift, and surface skipped/unparsable-file counts through existing progress reporting.
    • Data mapping to verify explicitly: workspace/solution extraction, timestamp fidelity, model identifiers, agent/tool-call events, and whether code blocks are structured enough to feed EditLocIndex reliably.

    Suggested checklist:

    • Confirm/store anonymized Visual Studio 2026 session samples
    • Verify discovery paths for persisted store and Temp-log fallback
    • Define Session mapping and failure behavior for partial/unparsable files
    • Add Visual Studio harness classification and dashboard/source gating coverage
    • Add parser tests for both happy-path and schema-drift cases

    Generated by Agentic Triage for issue #283 · 39.1 AIC · ⌖ 14.9 AIC · ⊞ 21.4K · ◷

    Add this agentic workflows to your repo

    To install this agentic workflow, run

    gh aw add githubnext/agentics/workflows/issue-triage.md@df35cf29fb856d1c3b8f023ed46d19126e7813bf
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions