Skip to content

E8: run VM density + saturation (both arms) and record the result #11

Open

Description

@mrsabath

Summary

Run E8 — VM density and saturation (P6 §5.2) — and record the result. The spec amendment and driver
work are #10; this issue is the run itself, which #10 explicitly does not cover.

Splitting them is deliberate. #10 lands a two-arm methodology (capacity + realism) and the driver
changes that make a knee reachable; it can be reviewed and merged without any hardware booked. The run
consumes that methodology, produces the number, and is where a bound=not-observed outcome has to be
adjudicated rather than merged. Keeping them in one issue is what let the previous calibrated run be
recorded as a density figure when it was stub-sleep-bound.

Preconditions — all of them, checked before booking anything

# Gate Why it blocks
1 #10 merged Without the capacity arm the ladder tops out at the admission cap W×S=32, a config choice, and the run reproduces bound=not-observed
2 PR rossoctl/moca#253 merged /turn's tool calls must reach the sandbox pool. A run where tools execute locally measures a harness with no hands tier — and the counting probe is what catches it
3 rossoctl/moca#239 merged if mixed-subject rungs are claimed P6 §3.6/§7: single-subject rungs exercise every mechanism this slice adds and need no P5. Mixed subjects and the isolation claim need it. Decide which run this is before starting
4 Model stub deployed with a tool-call profile (§5.4) A text-only stub means no session reaches the sandbox and the density number silently excludes the hands tier
5 Duty basis pinned to one row of §2.3 Default E6/OCP: duty = 0.061–0.079, provision at 12.6–16.5. The driver asserts K ≥ ceil(W × S × duty) with duty and the ratio from the same row

Gate 3 is a scoping decision, not a dependency to wait on. Per P6 §3.6, P5 slipping delays a claim,
not the work — so a single-subject E8 is runnable the moment gates 1, 2, 4, 5 hold, and should not be
held behind rossoctl/moca#239.

What to run

Both arms from #10's amended §5.2, each recorded separately and never merged into one figure:

  • capacity arm — trivial exec, fast stub profile, sandbox tier non-binding and non-competing.
    Sweep offered concurrency upward with SH_WORKERS × SH_TURNS_PER_WORKER and
    KAGENTI_SANDBOX_CAP set well past the expected knee. Reports an upper bound on the harness
    tier
    — explicitly not a deployable density.
  • realism arm — calibrated duty (e6-ocp basis), stub profile 300/12/64. This is the deployable
    number, sandbox-shaped by construction.

W=1 is a rung, not an aside: it makes "one process would be simpler" a data point (§5.2).

Knee detection reuses E6's sustained-decline detectKnee with degradeX=2 against a warm C=1
baseline, and the knee is reported as a floor, not a ceiling (EXPERIMENTS.md:120).

Acceptance

  • Both arms run; each arm's stub profile recorded and pinned via assert_stub_pinned, and the
    arm label verified against what /profile returns
  • Per rung: throughput, p50/p95, event-loop lag p99, RSS, per-turn worker CPU, sandbox-container
    CPU, lease saturation, over-admission events, spurious 429s
  • Duty basis row and connections per session recorded on every rung, not only sticky ones
  • The knee is attributed — worker CPU vs RSS vs replenishment vs admission cap — not just located
  • If the capacity arm still returns bound=not-observed, say so and name what bound first; do not
    report a floor as a ceiling
  • % of ideal throughput is not reported as a capacity figure when the ideal is stub-sleep-bound
  • Single-subject vs mixed-subject scope stated in the record, with the isolation claim made only
    if P5: session identity isolation — per-request subject, no ambient credential moca#239 landed
  • Results in deploy/vm/EXPERIMENTS.md, pointer added from docs/experiment-results.md (§5.6)

Open question to settle during the run

Is event-loop lag trustworthy as the capacity arm's headline diagnostic? It read ~11 ms flat at
every rung including c=1, where the tier is nearly idle — which looks like a sampling or timer
artefact. §5.2 makes it the metric that attributes a knee to worker CPU, so verify it against a
deliberately CPU-starved worker before relying on it. If it cannot be trusted, per-turn worker CPU
(#254 §2e) carries the attribution alone and the record should say so.

Related: #10 (spec + driver), PR rossoctl/moca#253 (routing prerequisite), rossoctl/moca#239 (mixed-subject scope), E9 run.

Activity

  1. transferred this issue fromrossoctl/mocaon Oct 2, 2026
  2. pdettori commented on Oct 2, 2026

    @pdettori
    Member

    Transferred from rossoctl/moca on 2026-10-02, along with the other experiment issues, because their drivers moved here on 2026-09-25 (deploy/microvm/EXPERIMENTS-MOVED.md, deploy/knative/EXPERIMENTS-MOVED.md, experiments/MOVED.md in moca).

    Bare #N references in this issue's body and earlier comments point to rossoctl/moca, not to this repo. GitHub doesn't rewrite them on transfer. For example, #253 means rossoctl/moca#253. The issues moved together, old number to new:

    rossoctl/moca here title
    #130 #2 SWE-bench measurement driver polish
    #256 #3 P4 next-experiments epic
    #257 #4 real model think-time distribution
    #261 #5 E11 with a real model in the loop
    #262 #6 test the admission gate (sealed prediction 1)
    #280 #7 E13 root cause
    #306 #8 rename coldAcquireRate
    #298 #9 E11 bash/Go cause-vocabulary drift guard
    #254 #10 E8 capacity/realism arms
    #275 #11 E8 run
    #276 #12 E9 run

    Product-code issues they reference stay in moca, for example rossoctl/moca#274 (per-Exec VM lifetime), #277 (P4.1 egress), #297 (vmpool test flakes), #307 (Destroy cost) and #338 (workspace lifetime).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions