Skip to content

improve(chat): make tool activity compact and fully inspectable - #1092

Merged
bkudiess merged 13 commits into
openclaw:mainfrom
calebeden:calebeden-improve-tool-call-presentation
Aug 6, 2026
Merged

bkudiess merged 13 commits into
openclaw:mainfrom
calebeden:calebeden-improve-tool-call-presentation

Conversation

@calebeden

@calebeden calebeden commented Aug 5, 2026 •

Copy link
Copy Markdown
Contributor

What Problem This Solves

Consecutive agent tool calls can overwhelm the tray chat transcript with large standalone cards, making tool-heavy responses difficult to scan. The prior presentation also made complete input and output details harder to inspect without losing conversation context.

Why This Change Was Made

Consecutive tool calls now render as one compact activity disclosure. Expanding the activity reveals dense per-tool rows, and expanding an individual row reveals its bounded input and output details.

The implementation preserves source chronology and call identity across live updates and saved-history replay. Structured result blocks prefer their semantic call-reference fields over generic block IDs, and aborted tool-first assistant history is replayed as one stopped response rather than leaking into a later message.

Grouping, expansion state, rendering, and replay projection are handled by focused owners so the compact presentation remains stable across mixed text and tool transcripts.

User Impact

Users can:

  • Scan substantially more tool activity without losing the surrounding conversation.
  • Expand one activity group instead of navigating many full-size cards.
  • Inspect individual tool names, running, success, error, and interrupted states, inputs, and outputs.
  • Select and copy visible input and output text.
  • Keep text and tool events in their original order when loading historical conversations.
  • Reload saved tool activity without orphaning results when a result block contains multiple identifier fields.

Evidence

Current PR head: c0b7b36978dd7403b275e3aab79718918d0c978e

The final history-correlation repair does not change the visual renderer. These screenshots remain representative of the compact and expanded presentation at the current head.

Collapsed tool activity

Screenshot 2026-08-04 182304

Single tool fully expanded

Screenshot 2026-08-04 182507

Current-head automated proof verifies that the production Reactor renderer starts collapsed, exposes accessible expansion state, lazily realizes nested tool rows, renders selectable output only after expansion, and removes nested rows when re-collapsed. Current-head history tests directly exercise semantic result correlation, mixed text and tool chronology, string-valued tool results, and aborted tool-first assistant replay.

Change Type

  • Bug fix
  • Feature
  • Refactor
  • Docs or instructions
  • Tests or validation
  • Security hardening
  • Chore or infrastructure

Scope

  • Tray or WinUI UX
  • Windows node capability
  • Local MCP or winnode
  • Gateway, connection, or pairing
  • Setup or onboarding
  • Permissions, privacy, or security
  • Tests, CI, or docs

Validation

Local validation at c0b7b36978dd7403b275e3aab79718918d0c978e:

  • ./build.ps1: passed
  • dotnet test ./tests/OpenClaw.Shared.Tests/OpenClaw.Shared.Tests.csproj --no-restore: 3,412 passed, 32 skipped, 0 failed, 3,444 total
  • dotnet test ./tests/OpenClaw.Tray.Tests/OpenClaw.Tray.Tests.csproj --no-restore: 2,165 passed, 0 skipped, 0 failed, 2,165 total
  • Focused structured result-reference parsing coverage: 4 passed, 0 failed
  • Focused saved-history correlation and abort replay coverage: 6 passed, 0 failed
  • ReactorToolActivityProofTests.ConsecutiveTools_RenderCompactActivityWithAccessibleExpansion: passed
  • Final read-only correctness review: no significant findings

Real Behavior Proof

  • Environment tested: Windows 11 ARM64 locally and Windows x64 automated UI runtime
  • PR head or commit tested: c0b7b36978dd7403b275e3aab79718918d0c978e
  • Exact steps or command run: Built the current head, ran the full Shared and Tray suites, ran focused saved-history parsing and replay tests, and ran the production Reactor activity proof. The developer also manually verified that the collapsed and expanded screenshots remain representative of the current presentation.
  • Evidence after fix: The screenshots show the compact default state and a fully expanded individual tool. The current-head Reactor proof verifies collapsed, expanded, nested-detail, selectable-output, and re-collapsed behavior. Focused provider tests replay structured call/result history through the tray data provider and verify chronology and correlation in the resulting timeline.
  • Observed result: Consecutive tools occupy one compact row by default. Expanding the activity reveals dense tool rows, and expanding one tool exposes readable, selectable details without exposing arbitrary argument payloads. Saved tool results complete the correct call when semantic and generic identifiers differ.
  • Screenshot or artifact links verified? (Yes/No/N/A): Yes
  • Not verified or blocked: A manually captured current-head tray session that reloads the conflicting-ID saved-history fixture is not available. The behavior is covered through the current-head parser and tray-provider replay tests.

Security Impact

  • New permissions or capabilities? (Yes/No): No
  • Secrets or tokens handling changed? (Yes/No): No
  • New or changed network calls? (Yes/No): No
  • Command or tool execution surface changed? (Yes/No): No
  • Data access scope changed? (Yes/No): No
  • If any answer is Yes, explain the risk and mitigation: N/A

Expanded rows display only allowlisted tool argument keys. Command-like values pass through the existing display sanitizer, arbitrary JSON and environment payloads are omitted, internal correlation identifiers are not rendered, and displayed values are bounded.

Compatibility and Migration

  • Backward compatible? (Yes/No): Yes
  • Config or environment changes? (Yes/No): No
  • Migration needed? (Yes/No): No
  • If yes, list the exact upgrade steps: N/A

Review Conversations

  • I replied to or resolved every bot review conversation addressed by this PR.
  • I left unresolved only conversations that still need maintainer judgment.

calebeden and others added 6 commits August 4, 2026 15:28
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 session-state 🚨 Merging this PR could lose, corrupt, stale, or mis-associate session or agent state. labels Aug 5, 2026
@clawsweeper

clawsweeper Bot commented Aug 5, 2026 •

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 6, 2026, 2:42 AM ET / 06:42 UTC.

ClawSweeper review

What this changes

The PR groups consecutive tray-chat tool calls into compact expandable activity rows and preserves structured tool call/result order when replaying saved chat history.

Merge readiness

⛔ Blocked until stronger real behavior proof is added - 4 items remain

Keep open. The supplied patch appears cohesive and has no supported line-level correctness finding, but the final saved-history correlation repair lacks current-head real tray proof; the PR itself says the conflicting-ID replay was not manually captured.

Priority: P2
Reviewed head: c0b7b36978dd7403b275e3aab79718918d0c978e

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The patch has substantial focused coverage and no supported correctness finding, but its real-proof package does not demonstrate the final current-head replay repair.
Proof confidence 🦪 silver shellfish (2/6) Needs stronger real behavior proof before merge: Screenshots directly show the compact visual state, but they do not show the current-head conflicting-ID saved-history replay introduced by the final commit; add redacted native tray evidence for that path before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Needs proof Needs stronger real behavior proof before merge: Screenshots directly show the compact visual state, but they do not show the current-head conflicting-ID saved-history replay introduced by the final commit; add redacted native tray evidence for that path before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 5 items Current main behavior: The current Reactor timeline explicitly orders entries for presentation before rendering; the PR replaces that route with chronological activity grouping, so this remains a meaningful unmerged change rather than work already on main.
Architecture boundary: The architecture ledger assigns native tool identity and allowlisted display projection to NativeToolProjector and directs timeline rendering toward a focused ToolCallCardRenderer, matching the PR's intended split.
Current-head proof gap: The PR body identifies c0b7b36 as the tested head and supplies visual screenshots plus tests, but expressly says a current-head tray session reloading the conflicting-ID history fixture is unavailable. The screenshots therefore do not directly demonstrate the final history-correlation repair.
Findings None None.
Security None None.

How this fits together

Gateway chat history is parsed into ordered message content, replayed into the tray timeline, then rendered as WinUI chat rows. This change affects both that replay boundary and the expandable presentation of tool activity.

flowchart LR
A[Gateway chat history] --> B[History parser]
B --> C[Replay projection]
C --> D[Chat timeline]
D --> E[Tool activity grouping]
E --> F[Expandable tray chat rows]
Loading

Before merge

  • Add real behavior proof - Needs stronger real behavior proof before merge: Screenshots directly show the compact visual state, but they do not show the current-head conflicting-ID saved-history replay introduced by the final commit; add redacted native tray evidence for that path before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Resolve merge risk (P1) - Merging changes the saved-history correlation and ordering boundary; tests do not replace a current-head tray replay that visibly proves a result with conflicting identifiers completes the intended tool row.
  • Resolve merge risk (P1) - The PR exposes parsed tool-history details through a new renderer, so the existing allowlist, bounded display values, and sanitization boundary need real-path confirmation in addition to unit coverage.
  • Complete next step (P2) - The required evidence must come from the contributor's current-head tray environment, so no safe automated repair lane can create it.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Patch scope 25 files, +3,235 / -263 lines The change spans shared history parsing, tray replay, rendering, localization, and UI tests, so the final end-to-end proof should cover the integrated path.
Production versus tests production +1,783, tests +1,452 The substantial test addition is useful, but the remaining merge gate is real behavior evidence for the final replay path.

Merge-risk options

Maintainer options:

  1. Add current-head history replay proof (recommended)
    Capture a redacted tray run at the current head that reloads a structured tool call/result fixture with differing semantic and generic IDs and shows the result attached to the intended activity row.
  2. Accept test-only replay evidence
    Maintainers may merge on the parser and provider tests alone, accepting that the final replay correlation has no current-head visible tray proof.

Technical review

Best possible solution:

Keep the focused renderer and replay projection, then merge only after a current-head native tray run reloads a redacted conflicting-ID history fixture and visibly shows the correct grouped tool result.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a feature PR. The focused fixture is covered by parser and provider tests, but the final behavior still needs a current-head real tray replay demonstration.

Is this the best way to solve the issue?

Yes for the overall direction: the focused activity renderer and replay projection fit the recorded ownership boundaries. The proof package is not yet the best merge evidence because it does not visibly exercise the last correlation repair.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against cdcb74f44a21.

Labels

Label justifications:

  • P2: This is a user-visible tray-chat improvement with bounded impact on transcript rendering and saved-history replay.
  • merge-risk: 🚨 session-state: The PR changes how persisted chat tool calls and results are correlated and ordered during replay.
  • merge-risk: 🚨 security-boundary: The new detail renderer carries untrusted tool-history values to the UI and must preserve the existing display filtering boundary.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🐚 platinum hermit.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs stronger real behavior proof before merge: Screenshots directly show the compact visual state, but they do not show the current-head conflicting-ID saved-history replay introduced by the final commit; add redacted native tray evidence for that path before merge. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. Screenshots directly show the compact visual state, but they do not show the current-head conflicting-ID saved-history replay introduced by the final commit; add redacted native tray evidence for that path before merge.

Evidence

What I checked:

  • Current main behavior: The current Reactor timeline explicitly orders entries for presentation before rendering; the PR replaces that route with chronological activity grouping, so this remains a meaningful unmerged change rather than work already on main. (src/OpenClaw.Tray.WinUI/Chat/ReactorChatTimeline.cs:199, cdcb74f44a21)
  • Architecture boundary: The architecture ledger assigns native tool identity and allowlisted display projection to NativeToolProjector and directs timeline rendering toward a focused ToolCallCardRenderer, matching the PR's intended split. (docs/ARCHITECTURE.md:62, cdcb74f44a21)
  • Current-head proof gap: The PR body identifies c0b7b36 as the tested head and supplies visual screenshots plus tests, but expressly says a current-head tray session reloading the conflicting-ID history fixture is unavailable. The screenshots therefore do not directly demonstrate the final history-correlation repair. (c0b7b36978dd)
  • Feature history: Current-main history identifies the Reactor timeline migration, native tool identity preservation, and recent history work as the relevant ownership trail. (src/OpenClaw.Tray.WinUI/Chat/ReactorChatTimeline.cs, 9acafa606a4d)
  • Inspection limitation: The checkout contains current main but not the PR head object. Reading it attempted a promisor fetch that failed because GitHub DNS was unavailable, so exact-head source review relies on the supplied PR diff and body.

Likely related people:

  • karkarl: Karen authored the current main Reactor chat migration that owns the production timeline being changed. (role: introduced the production Reactor chat timeline; confidence: high; commits: 9acafa606a4d; files: src/OpenClaw.Tray.WinUI/Chat/ReactorChatTimeline.cs)
  • Barbara Kudiess: Barbara authored the current-main native tool identity work that defines the allowlisting and display-sanitization boundary this renderer must retain. (role: recent native-tool projection contributor; confidence: high; commits: ad472243bcb8; files: src/OpenClaw.Tray.WinUI/Chat/NativeToolProjector.cs, src/OpenClaw.Tray.WinUI/Chat/OpenClawChatDataProvider.cs)
  • Caleb Eden: Caleb authored the recent current-main compacted-history change and is also the PR author, giving relevant prior merged-history context. (role: recent chat-history contributor; confidence: medium; commits: d2de6150753b; files: src/OpenClaw.Tray.WinUI/Chat/OpenClawChatDataProvider.cs)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Capture a redacted current-head tray session that reloads the conflicting semantic/generic-ID history fixture and visibly confirms the correct tool result correlation.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (14 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-05T21:28:34.975Z sha 959ceb4 :: needs real behavior proof before merge. :: [P2] Extract structured history parsing from the gateway client
  • reviewed 2026-08-05T21:39:12.016Z sha 959ceb4 :: needs real behavior proof before merge. :: [P2] Extract structured history parsing from the gateway client
  • reviewed 2026-08-05T22:13:57.161Z sha ca6f326 :: needs changes before merge. :: [P2] Preserve call IDs for string-valued tool results
  • reviewed 2026-08-06T03:36:51.509Z sha e65f771 :: needs changes before merge. :: [P2] Preserve call IDs for string-valued tool results
  • reviewed 2026-08-06T04:06:00.489Z sha e72e6a2 :: needs changes before merge. :: [P2] Parse string-valued tool results before returning
  • reviewed 2026-08-06T05:04:43.867Z sha 8956667 :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-06T05:21:02.792Z sha 8956667 :: needs real behavior proof before merge. :: none
  • reviewed 2026-08-06T06:28:19.080Z sha c0b7b36 :: needs real behavior proof before merge. :: [P2] Sanitize names from structured history tool calls

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. labels Aug 5, 2026
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. and removed proof: sufficient Contributor real behavior proof is sufficient. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. labels Aug 5, 2026
calebeden and others added 2 commits August 5, 2026 14:23
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added the proof: sufficient Contributor real behavior proof is sufficient. label Aug 5, 2026
@clawsweeper clawsweeper Bot added status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 5, 2026
…l-activity

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

# Conflicts:
#	src/OpenClaw.Tray.WinUI/Chat/OpenClawChatDataProvider.cs
#	src/OpenClaw.Tray.WinUI/Chat/ReactorChatTimeline.cs
@clawsweeper clawsweeper Bot removed the proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. label Aug 6, 2026
calebeden and others added 2 commits August 5, 2026 21:01
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Aug 6, 2026
Prefer semantic tool-result call references and consume aborted assistant state on the first replayed part, including tool-first messages.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Aug 6, 2026
@bkudiess
bkudiess merged commit aac07ab into openclaw:main Aug 6, 2026
18 checks passed
@calebeden
calebeden deleted the calebeden-improve-tool-call-presentation branch August 6, 2026 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 session-state 🚨 Merging this PR could lose, corrupt, stale, or mis-associate session or agent state. P2 Normal priority bug or improvement with limited blast radius. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants