Skip to content

benchmarking: actor telemetry for the sweperf workload - #1848

Merged
Haven Xia (HavenXia) merged 1 commit into
agent-substrate:mainfrom
Nishanth29:benchmarking/actor-telemetry
Sep 30, 2026
Merged

Haven Xia (HavenXia) merged 1 commit into
agent-substrate:mainfrom
Nishanth29:benchmarking/actor-telemetry

Conversation

@Nishanth29

@Nishanth29 Nishanth Kotla (Nishanth29) commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

benchmarking: actor telemetry for the sweperf workload

What this does

Adds the suspend/resume and task-execution timings from the telemetry proposal to the sweperf boomer workload.

New locust rows

row what it measures
ResumeToFirstExec resume RPC start until the sandbox accepts the cycle's /execute
TaskCEL in-container execution time for one sweperf task (4 cycles), as reported by replay.py
CycleCEL in-container execution time for one cycle
TaskWallClock client wall clock for one task, excluding think time
<rpc>_rtt client round trip for each control-plane RPC

The four derived rows are recorded under the actor method, the _rtt rows under grpc.

We keep both the client RTT and the server-side elapsed for every control-plane RPC so the network and queueing overhead stays visible separately. The existing ResumeActor / SuspendActor rows keep the server-side elapsed from the response trailer.

The liveness check at session start leaves the actor running, so the first cycle's resume is a no-op. Its successful ResumeActor, ResumeActor_rtt and ResumeToFirstExec samples are skipped (failures are still recorded), the same way ateapi skips no-op resumes.

Poll interval

The /status poll interval is now configurable with --sweperf-poll-interval-ms (env LOCUST_SWEPERF_POLL_INTERVAL_MS), default 100 ms (was a fixed 25 ms). It flows through dynconfig like the other --sweperf-* flags and is read per job, so a mid-run change applies from the next cycle.

Documentation

benchmarking/README.md had no sweperf section, so this adds one listing every row the workload emits and the poll interval flag, matching the existing DurDir section.

Testing

go test -race ./internal/benchmarking/boomer/..., plus runs on a real cluster (sympy, 4 users / 2 workers, gVisor): 5 min at the default 100 ms poll interval and 2 min at 250 ms.

  • Both had 0 failures and all expected rows.

  • The min ResumeActor is 334 ms, so the first-cycle no-op is no longer recorded.

  • ResumeActor / SuspendActor counts match (197 / 197), and each _rtt is within ~1 ms of its server-side row.

  • Client overhead over CEL per cycle is 65 ms at 100 ms vs 141 ms at 250 ms, so the flag takes effect.

  • Tests pass

  • Appropriate changes to documentation are included in the PR

@Nishanth29

Copy link
Copy Markdown
Contributor Author

@bowei Bowei Du (bowei) added kind/feature An enhancement / feature request or implementation area/benchmarking area/observability labels Sep 24, 2026
Comment thread benchmarking/README.md Resolved
@Nishanth29

Copy link
Copy Markdown
Contributor Author

Mauricio Poppe (@mauriciopoppe) thanks for the review! Addressed your comments in 75b3633, ptal when you get a chance.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please squash your commits into 1. Is there a trace you could share to see the result?

@mauriciopoppe

Copy link
Copy Markdown

/lgtm

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@maxsmythe Max Smythe (maxsmythe) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HavenXia
Haven Xia (HavenXia) added this pull request to the merge queue Sep 30, 2026
Merged via the queue into agent-substrate:main with commit 9e52546 Sep 30, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/benchmarking area/observability kind/feature An enhancement / feature request or implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants