feat(core) astubbs#227: auto scaling / adaptive concurrency - the engine discovers its own admission target

[astubbs]

depends on #335 - the atomic work claim
depends on #336 - the poller load gate by conservation
depends on #358 - dispatch parity counted, not timed
depends on #359 - residence time, the first latency metric
depends on #360 - virtual threads for the user function
depends on #361 - direct pull finished, scan made O(1)
depends on #362 - the bench harness and its results
depends on #363 - the campaign records and the integration branch
depends on #367 - the strategy-conversation notes this work inherits (the delta vote's WHY encoding, the shared-execution-resources design, the admission model)

Serves #227, and closes both halves of #311 along the way.

Base branch: this targets perf/engine-concurrency, not master. That branch is not in master
and has no PR of its own, and this work is grounded on the tree it produced - the direct-pull
engine, virtual threads, the conservation-derived load gate and the residence-time instrumentation
all changed the seam this feature governs. Targeting master would show that branch's entire history
as though it were part of this change. This cannot merge before its base does; retargeting to
master is a one-flag change if that is preferred.

Description

maxConcurrency is a compile-time guess about a runtime quantity. Too low silently strands
throughput nobody files an issue about; too high floods downstream services. The right value depends
on the data currently in the instance's assigned partitions, differs between instances in the same
group, and shifts with the workload - so no static number stays correct.

This adds an opt-in adaptive controller that discovers it. When enabled, the controller governs the
admission target - the records the engine allows in flight - from measured service time, outcome
signals and sustained achieved in-flight, stepping up while performance holds and contracting when it
degrades, always beneath an effective maximum. The thread pool stays fixed at the ceiling; admission
is the control variable, which is the one knob that works identically on a platform-thread pool and
under virtual threads.

The mode is off by default and this is staged deliberately. OBSERVE computes and reports every
decision without acting - it is the diagnostic an operator can turn on to see what concurrency the
engine would pick and why it is not moving, at zero behavioural change. ENFORCE lets it act.
Promoting one to the other is a restart, and the system property exists for the bench harness and CI
matrix but may select at most OBSERVE - an ambient JVM-wide flag must never be able to hand an
experimental controller the admission target of a production instance.

What the design review changed

A five-persona review plus an architecture pass ran against the plan before implementation and found
several things that would have shipped as bugs. Each of these is now a test, most of them
sabotage-proven red before restore:

• The pinned load factor would have doubled the enforced target. The factor means keep the
  workers N deep in buffered work, which held only while pool size equalled the target. With the
  pool at the ceiling and a live target below it, target x factor records would have run, not
  buffered. Dispatch now consumes the target un-multiplied; the factor applies only to the poller
  gate's buffer arithmetic.
• Starting unseeded at the effective maximum would have raised concurrency on opt-in - exactly
  the flooding shape the feature exists to prevent - because the ceiling substitutes upward when the
  user left maxConcurrency alone. The unseeded start is now the target today's configuration
  derives, and the controller ramps into the headroom from there.
• A fast-rejecting overloaded downstream reads as improved latency, so a latency gradient alone
  grows the target and accelerates the flood. A failure-fraction inhibitor freezes growth
  independently of what latency says.
• A contraction narrows the buffer and can manufacture its own starvation evidence, which would
  then freeze the controller in a one-way ratchet. A starved window below the ceiling earns a bounded
  probe up instead.
• The drain release had nowhere to fire from: an edge action on the draining transition runs on
  the caller's closing thread while the tick is already gated off. It is a state-derived seam read, so
  a contracted target cannot stretch a drain past its timeout - including when close arrives before
  the control loop ever ran.
• A cooperative rebalance fires callbacks on instances whose partitions did not move, so resetting
  on any callback would starve the controller of history in exactly the churning groups where
  per-instance adaptation matters most. The reset is gated on a real delta to this instance's
  assignment.

The control law, and what measurement settled

The math is Gradient2 with the upstream anti-drift fixes, ported from Netflix/concurrency-limits
(Apache-2.0, attributed per file) rather than taken as a dependency - the core stays
zero-dependency, and the upstream interfaces fit neither min-composed ceilings nor the sampling this
engine needs. Upstream ships no tests for that algorithm, so the port brought its own.

Two deviations are measured rather than preferred. The utilization term is the window median of
in-flight with a spread, not the maximum: under a skewed key distribution this engine sustains one
record in flight of a configured twenty-four while a maximum reads full width, so a maximum reports
health exactly when the engine is starved. And the port's own gate test settled a question the plan
left open - the ported law provably cannot descend when every sample it has ever seen comes from
an already-flooded operating point, so a bounded probe-down is implemented rather than deferred as a
known residual.

Scope

Core engine, pre-loaded-queue path only. The async engines follow their timing fix; the direct-pull
engine consumes no admission target at all and refuses the mode with a warning, as do the external
engines - refusals are loud, never silent. Instance-count recommendation, predictive scaling and any
distributed coordination substrate are explicitly out of scope.

Where ordering starves a workload, admission is not the binding constraint and no admission
controller can raise throughput. The controller's job there is to report rather than adapt against a
constraint it cannot move.

Same-defect sweep for #311

The defect class is rounding up to a multiple using the wrong operand. Every modulo in main code
was searched, untruncated: eight raw hits, of which one other is real ceiling arithmetic - the
buffer-size division in the module's load-factor construction - and it is a correct single-pair
ceiling division, not the wrong-third-variable shape. Single site, not a pattern.

Still to come

The adaptive benchmark arm, which lands on its own branch cut from the arrival-controlled harness -
the value claim is only measurable below saturation, and the roadmap stage is gated on that result
rather than on this PR. That is also why the roadmap entry here moves to in-progress rather than
implemented: that ladder reserves implemented for work proven in use.

Checklist

• Docs updated - operator-facing README section (regenerated from the template) and the metrics table
• User-facing feature documentation data added under docs/features/
• Tests added/updated
• Title & body reflect the final content of this PR

Stack position

This PR targets the perf/engine-concurrency integration branch, whose content the perf train is landing on master wagon by wagon (docs/inflight/branch-engine-concurrency-pr-stack.md is the wagon ledger). This PR retargets onto master once the train has landed; the depends-on list at the top is every cut wagon, kept current as the train moves.