Skip to content

Glutton Benchmarking Enhancements - #1631

Open
Max Smythe (maxsmythe) wants to merge 1 commit into
agent-substrate:mainfrom
maxsmythe:bm-benchmarking
Open

Glutton Benchmarking Enhancements#1631
Max Smythe (maxsmythe) wants to merge 1 commit into
agent-substrate:mainfrom
maxsmythe:bm-benchmarking

Conversation

@maxsmythe

Copy link
Copy Markdown
Collaborator
Benchmarking: multi-actor glutton VUs, live window, resume retries, client-side latency

Glutton actors per VU. Each VU creates --actors-per-user actors on
startup and cycles through them round-robin, so one goroutine can drive
many mostly-idle actors. runner.py forwards the flag to boomer-glutton.
A crashed actor stays crashed for the run (ateapi never rehabilitates
it), so it is marked on the first Aborted "crashed" error, skipped from
then on, and counted in a CrashCount stat.

Live window. --min/--max-live-time (default 0-0) set how long an actor
stays resumed between its first ping and the suspend. Up to
--max-pings-per-wake pings (default 1) run inside it, spaced 0.2-1.0s
apart. The wait window (--min/--max-wait-time) remains the gap between
one actor's suspend and the VU's next resume, and every return path
sleeps it, so a failing startUser, resume, or crashed actor does not
spin on boomer's zero-delay re-entry. With the defaults the cycle is
resume, ping, suspend, wait: the same shape as before.

Resume retries. ResumeActor retries ateapi's transient "concurrent
update conflict" Aborted up to five times with a 50ms backoff, inside
the timed call, so the conflict no longer shows up as a failure.

Client-side latency. Every gRPC row in the locust stats now reports
client wall clock, which covers retries, queueing, and the network. The
server's elapsed-time trailer stays on the trace span only.

Worker robustness. The router HTTP client keeps up to 10000 idle
connections per host so each VU reuses its connection across wakes. A
failed dynconfig fetch after the first successful one keeps the last
fetched values instead of exiting the worker.

Orchestrator. util.run logs the duration of each shell command.

Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

  • Tests pass
  • Appropriate changes to documentation are included in the PR

…lient-side latency

Glutton actors per VU. Each VU creates --actors-per-user actors on
startup and cycles through them round-robin, so one goroutine can drive
many mostly-idle actors. runner.py forwards the flag to boomer-glutton.
A crashed actor stays crashed for the run (ateapi never rehabilitates
it), so it is marked on the first Aborted "crashed" error, skipped from
then on, and counted in a CrashCount stat.

Live window. --min/--max-live-time (default 0-0) set how long an actor
stays resumed between its first ping and the suspend. Up to
--max-pings-per-wake pings (default 1) run inside it, spaced 0.2-1.0s
apart. The wait window (--min/--max-wait-time) remains the gap between
one actor's suspend and the VU's next resume, and every return path
sleeps it, so a failing startUser, resume, or crashed actor does not
spin on boomer's zero-delay re-entry. With the defaults the cycle is
resume, ping, suspend, wait: the same shape as before.

Resume retries. ResumeActor retries ateapi's transient "concurrent
update conflict" Aborted up to five times with a 50ms backoff, inside
the timed call, so the conflict no longer shows up as a failure.

Client-side latency. Every gRPC row in the locust stats now reports
client wall clock, which covers retries, queueing, and the network. The
server's elapsed-time trailer stays on the trace span only.

Worker robustness. The router HTTP client keeps up to 10000 idle
connections per host so each VU reuses its connection across wakes. A
failed dynconfig fetch after the first successful one keeps the last
fetched values instead of exiting the worker.

Orchestrator. util.run logs the duration of each shell command.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant