Repository navigation
Conversation
- FleetTaskEnv wraps FleetEnvClient with task-oriented interface - Accepts task configs from export_training_tasks.py - Creates versioned environments on reset - Injects task prompt into observations - Executes verifier for reward computation on episode completion - Supports both sync and async step methods - Factory functions: make_fleet_task_env, from_json_file - Tests: 20 unit tests for init, specs, verifiers, factories 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The MCP images don't exist for all environment versions, causing FleetVersionNotFoundError when trying to create environments. Changing the default to None allows the Fleet SDK to use standard images which are available for all versions. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
FleetEnvClient.from_fleet() was not accepting data_key/data_version parameters, causing them to be passed through **kwargs to HTTPEnvClient which doesn't accept them. - Add data_key and data_version as explicit parameters - Pass them to fleet.make() - Update task_env.py to pass them separately 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fleet SDK expects data_key in "key:version" format, not as separate parameters. Updated from_fleet() to combine them before calling fleet.make(). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTTPEnvClient.reset() doesn't support seed parameter yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Increases default timeout from 15s to 60s for Fleet API calls. This prevents timeouts during environment initialization. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously reset() did partial work and reset_async() added tool fetching. Now reset_async() does all the work (including fetching tools) and reset() is just a sync wrapper that calls it via run_until_complete(). This ensures both methods return identical results including tools. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
MCP's call_tool() returns a CallToolResult Pydantic object, not plain text. This was causing ugly repr strings to be passed to agents like: "meta=None content=[TextContent(type='text', text='...')] ..." Now properly extracts: - Text content from result.content[].text - Tries JSON parsing for structured results - Falls back to structuredContent if available - Handles isError cases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests for: - FleetMCPClient._extract_tool_result(): - Single text content extraction - JSON parsing from text - Multiple text contents - Error result handling - Structured content fallback - Empty result handling - FleetTaskEnv reset: - reset_async() returns tools - reset() calls reset_async() (sync wrapper) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move fleet.make() and list_tools() into FleetTaskEnv.__init__() - Tools are now fetched at env creation, not during reset - reset_async() calls _orch.reset() with error handling, returns cached tools - Use asyncio.run() for Python 3.13 compatibility - Update tests for new initialization pattern 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Log task_key and verifier code preview when verifier fails - Catch syntax errors separately with clear message - Show which functions were found if 'verify' is missing Helps debug issues like "Verifier code must define a 'verify' function" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace custom _execute_verifier_local() with Fleet SDK's Task.verify_detailed() which properly sets up the verifier namespace with: - Environment type annotation - Helper functions (normalized_contains, etc.) - Proper function discovery (not just "verify" function) This fixes "name 'Environment' is not defined" errors during verifier execution. Changes: - _compute_reward: Create Fleet SDK Task and call verify_detailed() - Support both 'verifier_code' and 'verifier_func' field names - Add comprehensive logging for debugging - Remove broken _execute_verifier_local method Tests: - Update all verifier tests to mock Fleet SDK Task.verify_detailed() - Add tests for various edge cases (no verifier, no orch, exceptions) - Fix fixture to avoid asyncio.run() conflicts with pytest-asyncio 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add retry with exponential backoff (3 attempts, 1s/2s/4s delays) - Log errors instead of silently swallowing exceptions - Log warning when some clients fail but others succeed - Log error after all retries exhausted This fixes silent failures when MCP connections are flaky, which caused 'no tools found' errors in SkyRL training.
call_tool now retries with exponential backoff (3 attempts, 1s/2s/4s) on connection errors, similar to list_tools. ValueError (tool not found) is not retried. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds exponential backoff retry (3 attempts, 2s base delay) around fleet.make() to handle transient Fleet API errors like health check failures that can occur during instance provisioning. Only retries on transient errors (health check, timeout, connection). Permanent errors are raised immediately. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add Toolathlon-style context management tools for long trajectories: - check_context: Check visible/total turn counts - manage_context: Drop old turns to free up context space - search_history: Search all history (including dropped) - search_tool_output: Search truncated tool output - view_tool_output: Paginate through truncated output The ContextManager class can be used by any training framework that maintains chat_history. It tracks full history and handles truncated tool outputs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Computer-use tasks require MCP-enabled container images (e.g., famazon:mcp0.0.7) which have scrot installed for screenshots and the MCP server with 'computer' tool for mouse/keyboard control.
Previously, tools were only fetched for tool_use modality due to a restrictive condition. This caused computer_use tasks to fail with "no tools found in observation" because the computer tool (mouse, keyboard, screenshot) was never fetched. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When task_modality is computer_use, filter tools to only include the 'computer' tool. This prevents the model from using API tools when it should be using mouse/keyboard control. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Instead of calling Anthropic directly and running a local agent loop, the evaluator now: 1. Imports the generated task via fleet.import_task() 2. Creates a harness job via fleet.create_job() 3. Polls for completion 4. Extracts per-session verifier scores from job sessions Uses real Fleet model IDs (claude-sonnet-4.5, claude-opus-4.5) instead of the broken weak/strong mapping that required ANTHROPIC_API_KEY. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fleet harness POST /v1/jobs requires model IDs in 'provider/model' format (e.g., 'anthropic/claude-sonnet-4.5'), not just the model name. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The sync time.sleep() in _poll_job blocked the asyncio event loop, preventing trajectory timeouts from cancelling evaluations. Using asyncio.sleep() allows the event loop to properly handle cancellations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fleet returns session model IDs without provider prefix (e.g., "claude-sonnet-4.5")
while we configure them with prefix ("anthropic/claude-sonnet-4.5"). Added
_match_model_id() to normalize and match by bare model name, so scores land
in the correct results_per_model bucket.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
feat: Add TaskEvaluator for task generation inner loop
Add _tool_errors, _verifier_stdout, _verifier_error to FleetTaskEnv so SkyRL can build hints from failed rollout feedback without an LLM call. - Tool errors accumulated in step_async() on MCP errors and exceptions - Verifier stdout/error captured in _compute_reward() after verifier runs - Verifier exceptions also captured in _verifier_error (not just failures) - All feedback properties reset in reset_async() to prevent cross-episode leakage - Properties: verifier_stdout, verifier_error, tool_errors_list Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expose verifier feedback properties for hint generation
Add describe_db() and query_db() methods (sync + async) to FleetEnvClient, delegating to the Fleet SDK's SQLiteResource. This enables querying provisioned environment databases (seed/current) from outside the container via HTTP, which is needed for task generation workflows where the model explores DB data before designing tasks. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
describe_db_async/query_db_async were wrapping sync describe_db/query_db in asyncio.to_thread, but when _fleet_env is an AsyncEnv (from AsyncFleet.make), .db().describe() and .query() are async coroutines. Now detects whether the resource method is a coroutine and awaits directly instead of using to_thread. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When the Fleet MCP server returns {"base64_image": null} (failed screenshot),
_extract_tool_result would create an image_url block with url=None, causing
downstream AttributeError in SkyRL's extract_images_from_conversation.
Now returns a text error message instead of propagating the null.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
browser_use tasks now get the same treatment as computer_use: - MCP image type (triggers browser sidecar, no MCP image needed) - computer tool only (screenshot, click, type, scroll) - initial screenshot on reset - 1800s TTL Follows platform change: browsers are now sidecars on base images, MCP images are deprecated. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
browser_use tasks now create a standard Fleet env (web UI only) plus a separate browser lease via the Fleet browser API. The browser navigates to the env's web UI and provides the computer tool (screenshot, click, type, scroll) via its MCP endpoint. Flow: Fleet env (standard) → browser lease → navigate → healthcheck → use browser MCP for computer tool. On close, delete the lease. MCP images are deprecated — browsers are now sidecars on base images. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
FLEET_API_KEY is not accepted by the browser lease API (401). Use the shared fallback token from theseus api_token_config.py, same as the harness uses. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Both create_trace_job and upload_trace were hitting orchestrator.fleetai.com — that host runs the env-instance API, not the trace API, so every prior call silently 404'd (the "Failed to create Fleet trace job — Not Found" warning we'd been ignoring). Verified by hand against the live OpenAPI spec on 2026-06-16: POST https://api.internal.fleet-platform.fleetai.com/v1/traces/jobs POST https://api.internal.fleet-platform.fleetai.com/v1/traces/logs with Authorization: Bearer <api_key>. The previous SDK call also used X-API-Key, which the right host responds to with "Authentication required. Provide either API key or JWT token with team ID"; Bearer auth works for both the research and team keys. The /v1/traces/logs body schema also moved from the legacy {"messages": [{"role": "...", "content": "..."}]} shape to {"history": [{"role": "...", "parts": [{"type": "text"|"image", ...}]}]}. upload_trace now flattens OpenAI-style content arrays into the new parts shape so image blocks still surface as inline screenshots in the UI. End-to-end probe with both functions returns a real job_id and session_id (job 5c3831b6-..., session 544bab3a-...). Existing callers (SkyRL trainer.py, main_fleet_tinker.py) need no changes; their try/except wrappers swallow the old failure and will start uploading after this fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Empirically verified on 2026-06-16: Fleet's /v1/job-session-groups
aggregation reads score ONLY from sessions that have a linked
verifier_execution_id; the inline score field alone is stored but
ignored. Probe sequence:
- POST /v1/traces/logs {score: 1.0} → success, but avg_score in the
job-session-group response stays null
- POST /v1/traces/logs {score: 0.0, verifier_execution_id: <real id>}
→ group's avg_score flips from null to 0.0
So:
- FleetTaskEnv._compute_reward now stashes response.execution_id on
self._last_verifier_execution_id after every verify_detailed call.
- upload_trace gains a verifier_execution_id kwarg; forwards it to
/v1/traces/logs when present.
The SkyRL env wrapper (skyrl-fleet) reads the id from the OpenEnv handle
and threads it into upload_trace — covered by a companion commit there.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sing exec_id Root cause for all Tinker trace uploads landing with verifier_execution_id=null (symptom: Fleet dashboard showed "no grading runs"): the SDK's sync verify_detailed() spins up its own loop via asyncio.run() inside the asyncio.to_thread() worker, which collides with Event objects bound to the main loop. The verifier never completed server-side and the response we caught at except: was the loop-binding RuntimeError, so _last_verifier_execution_id was never set and the trace uploader sent no exec_id. Switching to fleet._async.tasks.Task and awaiting verify_detailed_async() in the harness's event loop makes the SDK use the async env's HTTP client directly, no thread-pool, no loop creation. Also: - 3-attempt retry with exponential backoff (3s, 6s) on 502/503/Bad Gateway/timeout from the env-instance reverse proxy. Fresh instances occasionally return 502 before they finish warming up; the next attempt succeeds. - Warn (not silently None) when execution_id comes back empty after a verifier call. Future regressions of the same symptom will be visible in logs. Verified against a real v6 bureau-tool_use task: returns success=True with populated exec_id, and trace upload links cleanly in the Fleet dashboard. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…uster) The signature dropped `cluster_name` — it's now auto-resolved by the central API at api.internal.fleet-platform.fleetai.com. task_env.py already calls with (instance_url, ttl_seconds) only, so the old `cluster_name`-required signature was failing every browser_use rollout with 'create_browser_lease() missing 1 required positional argument: cluster_name'. Auth: falls back FLEET_TEAM_API_KEY → BROWSER_API_TOKEN → FLEET_API_KEY. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BU/CU rollouts in canonical-v3 had 0/48 and 0/44 PASSes despite visible
per-app progress (BU 1/3 apps, CU 3/5 apps in many sessions). The
multi-app verifier wraps per-app blocks like:
<<< VERIFY_outlook <<<
... SUCCESS_ACCUMULATOR ['[C] Email sent', '[C] Email flagged'] ...
>>> VERIFY_outlook >>>
App outlook: 1
Combined result: 1/3 apps passed
and collapses across apps with binary `min()` -> the agent sees reward=0
even on rollouts that solved 2 of 3 apps. No gradient signal.
Parser now:
- Reads `App <name>: <0|1>` lines, returns n_pass / n_apps.
- Excludes trivial-pass apps whose SUCCESS_ACCUMULATOR is dominated by
"no unexpected changes" markers (medora on Lifeline-only tasks). They
don't represent solved work; counting them inflates fractional reward.
- Falls back to the legacy single-accumulator parser when no per-app
lines are present.
Opted into via `env_config.partial_reward=True`. Already plumbed through
skyrl-gym FleetTaskEnv.
Tests in tests/envs/test_fleet_partial_reward.py cover: real BU rollout
stdout with trivial medora, one/all real apps pass, all-trivial returns
None, three-app mix, legacy fallback, empty/random text.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported BU/CU screenshots not appearing in Fleet UI traces. Root
cause: /v1/traces/logs binds each `parts[*]` as a Gemini `Part` and only
reads `text`, `thought`, `inline_data` ({data, mime_type}),
`function_call`, `function_response`. The earlier shape
{"type": "image", "mime_type": "image/png", "data": "<base64>"}
pydantic-validates because every `Part` field is Optional, but
`part.inline_data` is None, so process_content's S3 upload step never
fires and the image silently never lands in the UI. (Text parts
coincidentally worked because the server reads `part.text` regardless of
the redundant `type` key.)
Switches to:
{"inline_data": {"mime_type": "image/png", "data": "<base64>"}}
Text becomes `{"text": "..."}` (no `type`) to match.
Server-side S3 upload of base64 image bytes is handled automatically by
`process_content` in theseus/orchestrator/core/session_content.py — no
client-side presigned-URL sidecar step needed.
Source of truth: theseus/orchestrator/private_api/traces.py
- TracePart (~line 152): only `text`, `thought`, `inline_data`,
`function_call`, `function_response` are read.
- _extract_content_from_parts (~line 431): pulls
`part.inline_data.data` and `part.inline_data.mime_type`.
Tests in tests/envs/test_fleet_trace_image_shape.py pin the shape (PNG
data URL, JPEG mime preservation, remote URL fallback, text block,
unknown block).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sessions/ingest
Previous path emitted Gemini-shape inline_data parts to /v1/traces/logs.
The server accepted those uploads but stored the base64 payload in the
PRIVATE bucket s3://theseus-model-traces, so every <img src=...> tag the
Fleet UI rendered returned HTTP 403. Screenshots were silently invisible
in the canonical hero run for both BU and CU, which the model reads as
"my click didn't land" and gives up after ~10 turns.
New path mirrors what nova/import_traces.py:upload_screenshot_to_s3 does
and what the legacy skyrl harness used to do:
1. For each multimodal image_url block carrying a data:image/...;base64
URL, the harness uploads the bytes directly to
s3://fleet-sessions-images/harness/screenshot/<sha256[:24]>.<ext>
via boto3. Bucket is public-read so the Fleet UI can <img src=https://...>
with no auth.
2. The image_url URL in the chat history is rewritten to that HTTPS URL.
Dedup is content-addressed: a second turn with the same screenshot
bytes produces the same S3 key, so re-uploads short-circuit on
head_object.
3. The rewritten chat history is POSTed to orchestrator/v1/sessions/ingest
as OpenAI multimodal shape (messages: [{role, content: [...]}]), NOT
the Gemini history/parts shape /v1/traces/logs wants. /v1/sessions/ingest
does NOT pre-upload images server-side; the public-URL rewrite must
happen client-side or the UI won't render anything.
S3 upload failures degrade gracefully — the block is left alone, the
session ingest still proceeds. We'd rather have a broken-image session
than crash a training rollout. AWS keys come from the existing container
environment (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) which already had
put_object permission on the public bucket; verified before this commit.
Tests rewritten to cover:
- data URL extraction (png/jpeg/malformed/already-https)
- content-addressed S3 key dedup
- chat history walk (image rewrite, text passthrough, string content,
S3 failure passthrough, already-HTTPS passthrough)
- upload_trace integration: endpoint is /v1/sessions/ingest, payload is
OpenAI shape with `messages`, no `history`/`parts` from the old code,
image URLs are HTTPS not data:, reward survives on metadata.score,
network failures return None without raising.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hestrator Regression from 6dc3fd1: I collapsed both endpoints to https://orchestrator.fleetai.com. /v1/sessions/ingest exists on orchestrator and the rewrite was correct for THAT call. But /v1/traces/jobs lives on the platform host (api.internal.fleet-platform.fleetai.com), NOT on the orchestrator — orchestrator's /v1/jobs is the eval-job endpoint and takes env_key/project_key, not a trace name. Consequence in production (job 390e7f72, the BU canonical refire just killed): main_fleet_tinker.py's `create_trace_job(api_key, trace_job_name)` got `404 Not Found` from POST orchestrator/v1/traces/jobs. The outer try/except logged the warning. set_trace_config() never ran. Every episode_done saw no trace config and skipped upload_trace silently. Training proceeded, WandB metrics logged, but zero Fleet UI sessions were created. Fix: two host constants. _FLEET_TRACE_JOBS_HOST = platform host → /v1/traces/jobs _FLEET_SESSION_INGEST_HOST = orchestrator → /v1/sessions/ingest Both endpoints share the underlying Supabase job pool — a job_id minted on the platform host is recognized by the orchestrator's session ingest AND rendered at fleetai.com/dashboard/jobs/<id> the same way. Tests: - Pin the EXACT URL in test_upload_trace_posts_openai_shape_to_sessions_ingest (`https://orchestrator.fleetai.com/v1/sessions/ingest`). - New test_create_trace_job_posts_to_platform_host pins the exact `https://api.internal.fleet-platform.fleetai.com/v1/traces/jobs` URL. - New test_create_trace_job_uses_different_host_than_upload asserts the two host constants are NOT equal. This single assertion would have caught the regression. There was no test for create_trace_job before this commit, which is exactly how the regression slipped through. Pinned now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ervations
Verified live with an A/B probe on 2026-06-20 (job 0ad01f6a):
two test sessions with identical content + same screenshot URL.
role:'user' rendered no image. role:'tool' + tool_call_id rendered the
screenshot inline as a thumbnail under the tool response panel.
The Fleet UI's SessionDetailView.tsx (added in cde0840 / PR #2293,
2026-04-09) has two transformer branches for content arrays:
if (m.role === 'tool') {
return JSON.stringify(m.content, null, 2); // ← image_url survives
}
if (Array.isArray(m.content)) {
const textParts = m.content
.filter((block: any) => block?.type === 'text' && typeof block?.text === 'string')
.map((block: any) => block.text);
if (textParts.length > 0) {
return textParts.join('\n'); // ← image_url stripped
}
}
So role:'user' content arrays get text-stripped and the screenshot is
lost. role:'tool' content arrays survive intact and
useSessionViewer.tryExtractFromToolResult extracts the image_url for
inline rendering.
FleetTaskEnv.step_async writes tool observations as role:'user' because
that's what Kimi's chat template binds. The model's chat_history MUST
keep role:'user' — renaming it would break the chat template. So this
rename happens ONLY at upload time, ONLY for messages whose content
carries an image_url block. The model trajectory is unchanged.
The renderer also keys off tool_call_id to look up the tool name. We
carry the most recent assistant.tool_calls[0].id forward. If missing
(Kimi inline tool calls often don't produce a structured tool_calls
list), synthesize a UUID so the UI has something stable to bind against.
Skill written up at fleet-ai/incantations/skills/trace-upload-with-images
so the next person doesn't re-discover this from scratch.
Tests pinned:
- test_role_mapping_user_with_image_becomes_tool
- test_role_mapping_carries_assistant_tool_call_id
- test_role_mapping_does_not_touch_user_text_messages
- test_role_mapping_preserves_tool_role_when_already_set
20 passing in test_fleet_trace_image_shape.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…t_history)
The env appends `{"type":"text","text":"[Turn N/M]"}` to every multimodal
observation so the model knows its trajectory position — this stays. But
the Fleet UI's session viewer renders a content array of
[image_url, footer_text] as siblings of a JSON tree, nesting the
screenshot inside the tree instead of showing it as primary content.
Stripping the footer block ONLY at upload time (trace._rewrite_chat_history)
gives:
- chat_history during inference: [image_url, "[Turn N/M]"] — model sees footer
- session ingest payload: [image_url] — UI renders image
The pattern matcher is robust to whitespace and spacing variants. Test
fixtures cover [Turn 3/64], " [Turn 12/64] ", "[Turn 64 / 64]", and
"[Turn 1/64]\n" — all stripped. Non-footer text blocks (search results,
other tool output) survive.
Verified locally with the VLM grounding smoke against a real Fleet
screenshot: build_model_input_chunks still produces an ImageChunk
(model still sees the image), Tinker accepts the multimodal payload,
Kimi-K2.6 returns a grounded response naming specific UI elements.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Fleet trace viewer keys off `assistant.tool_calls[0].id` matching
the next tool message's `tool_call_id` to link a screenshot back to
the call that produced it. Before this change, _assemble_messages's
forward branch did `messages.append({"role": role, "content": content})`
— stripping every field except role and content. Result: even when the
caller (skyrl-fleet env.py:step_async, after companion PR) sets
tool_calls on the assistant, the upload payload had it dropped and the
viewer saw nothing to link against. Tool Calls=0 in the viewer header,
screenshots rendered only inside the JSON tree.
A/B verified 2026-06-21 via dummy-job probe: identical session content
with tool_calls set on the assistant renders the screenshot top-level;
the same content without it doesn't. The forwarding gap was the causal
break in the pipeline.
This change preserves tool_calls when forwarding the assistant message
to /v1/sessions/ingest. Targeted, not a {**msg, ...} spread — we don't
want to accidentally forward unrelated fields a future caller might
attach to a message dict.
3 new tests pin the behavior:
- tool_calls round-trip on assistant (with tool_call_id linkage to the
next tool message)
- tool_calls not promoted onto non-assistant messages
- absent-tool_calls case stays absent (don't add empty list — could
confuse the viewer's linkage)
26 fleet trace tests pass.
Companion PR: skyrl-fleet PR #67 attaches tool_calls in env.py:step_async
after parse_tool_call succeeds. Both must land for the trace-viewer
linkage to work end-to-end.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Author
|
wrong repo — closing, re-opening on fleet-ai/OpenEnv fork |
This was referenced Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
_assemble_messagesnow preservestool_callson assistant messages when forwarding to/v1/sessions/ingest. Before this change, the forward branch didmessages.append({"role": role, "content": content})— stripping every field except role + content.Why
The Fleet trace viewer keys off
assistant.tool_calls[0].idmatching the next tool message'stool_call_idto link a screenshot back to the call that produced it. Withtool_callsdropped on upload, the viewer saw nothing to link against →Tool Calls=0in the header → screenshots only rendered inside the JSON tree, no top-of-session preview.A/B verified 2026-06-21 via dummy-job probe (5 trace jobs, one variant each): identical content with
tool_callsset on the assistant renders the screenshot top-level; the same content without it doesn't. This forwarding gap was the causal break.Targeted preservation, not a
{**msg, ...}spread — avoids accidentally forwarding unrelated fields a future caller might attach.Companion
skyrl-fleet PR #67 attaches
tool_callsinenv.py:step_asyncafterparse_tool_callsucceeds. Both PRs must land for the trace-viewer linkage to work end-to-end:tool_callsin chat_history → still dropped by trace.py → no visible changetool_callsif set → skyrl-fleet never sets it → no visible changeTest plan
pytest tests/envs/test_fleet_trace_image_shape.py -k tool_calls— 3/3 new tests greenpytest tests/envs/test_fleet_trace_image_shape.py— 26 passed (23 pre-existing + 3 new)Tool Calls=N+ top-of-session screenshot preview🤖 Generated with Claude Code