docs(perf): expression-evaluator codegen sweep — measured no-win

[akutuva21]

NO PRODUCTION CODE IS SHIPPED IN THIS PR. The diff is one new
documentation file, docs/PERF_EXPRESSION_CODEGEN_NO_WIN.md. There is no patch
here to review — this PR records a measured no-win so the next operator
does not repeat three experiments.

Verdict: measured NO-WIN. Nothing shipped.

bng::ast::Expression's per-node std::string dispatch is a real, measurable
cost in isolation, and the codegen fix for it genuinely works — but the
evaluator is not on any hot path in the product, so the fix buys nothing end to
end. The candidate is also ~9% slower at -O0. I implemented it, measured
it, could not show a win by any instrument that survived re-running, and
reverted it.

Compiler: Apple clang 21.0.0 (clang-2100.3.34.2), arm64-apple-darwin25.6.0.
Release flags unchanged from base; LTO on for bng_cpp in both arms. No
-ffast-math/-Ofast/-ffp-contract change, no tolerance change, no
#pragma GCC optimize, no target attributes.

Two retractions in this PR, one of them mine

1. The headline win, retracted. 935.48 → 714.89 ns/eval, "-23.6%", which I
reported to a peer before ever re-running it. Fifteen interleaved reps of the
same binaries in the same session inverted it to +4.78% median (slower)
against a 5.19% noise floor on the baseline itself.

2. The replacement figure, also retracted — corrected after
sciPkPd.PkSurvey and swarmMemory pressed on instrument claims. I published
B = +0.024% as though the instruction counter had resolved a difference. It
had not:

instr.txt    A_med 5,039,373,445  B_med 5,037,690,571   -0.033%  -> B FASTER
instr2.txt   A_med 5,037,345,446  B_med 5,038,543,790   +0.024%  -> B SLOWER

The sign flips between datasets, and the between-arm delta (~0.03%) is
smaller than the within-arm full range (0.061%–0.099% of median). The honest
reading is no measurable difference in either direction. Same class of
error as the 23.6%, one order of magnitude smaller, caught only because
swarmMemory checked their own counter's spread instead of trusting its label.

The no-win is unchanged and better supported for it: the candidate does not
merely fail to win, it is indistinguishable from baseline in a metric tight
enough to have caught a real difference.

Population table

Candidate	Change	Microbench	Instr @-O2	@-O0	Verdict
A	baseline	935.48 ns/eval	median 5,037,345,446	33,609	reference
B	hoist each literal's length ahead of its compare, 66 sites	714.89 (retracted)	flat — sign flips, delta below noise	36,618 (+9%)	rejected
C	B + factorial arm → [[gnu::cold]] [[gnu::noinline]]	792.06 (worse than B)	not measured	—	rejected

C was not pursued: it was built on B, and B did not survive re-measurement.

The codegen hypothesis was CORRECT — and it still didn't help

evaluateWithFunctions at -O2: 7,048 → 6,517 instructions (−7.5%), cmp
423 → 343, b 421 → 356, ldr 618 → 540. memcmp call sites 0 in both arms.
The length switch materialized exactly as predicted; the program does not get
faster, because the function is not on the hot path.

Why: the evaluator is cold, and the budget is ~5%

Four end-to-end sample profiles of an 8-reaction ODE where every rate law
references an observable or time: grep -c "Expression::evaluateWithFunctions"
-> 0, 0, 0, 0. The only engine frame present is OdeIntegrator::writeOutputFiles.

Ceiling — functional vs constant rate laws, 5 interleaved reps, 400k-step ODE:
delta mean 0.132 s, stdev 0.091 s = 4.89% of wall clock, noise
floor 2.67%. That is the absolute maximum any evaluator optimization can win
here, already below the ~11% host-drift floor.

End-to-end A/B, 10 interleaved reps: baseline min 2.500 / median 3.045;
candidate min 2.610 / median 3.080 → +4.40% min, +1.15% median, against a
10.21% baseline noise floor. Not resolvable.

Measurement context

• Runnable count R=21, with four named timing-sensitive processes live
  (ps -axo pid,etime,command).
• Loadavg band 38–102 (product A/B), 70–130 (opt sweep) — a band, not a
  decimal; memWatch proved vm.loadavg does not respond to a known
  single-core input here.
• Interleaved in one session: yes, every A/B.
• Peak RSS (/usr/bin/time -l): microbench 1,540,096 B · 400k-step ODE
  252,264,448 B · 4M-step ODE 492,765,184 B · ctest -j4 635,715,584 B.
• Metric class: instruction counts are the exempt class — measured
  insensitive to concurrent load, as measured, NOT "deterministic" (CPU
  migration, frequency scaling and core placement were untested, and the
  counter's own 0.06–0.10% spread sets a ~0.1% resolution floor). Every
  wall-clock number in this PR is unexempted and unclaimed.

Scope of this negative result

Supports exactly one claim:

On the C++ ODE path with functional rate laws, bng::ast::Expression is
absent from end-to-end profiles and the addressable budget is ~5%. No codegen
change to it can be worth landing on that path.

Not covered — an open item, not a cleared one. The only callers outside
Expression.cpp are NFsim's:

grep -rn "evaluateWithFunctions" cpp/ --include=*.cpp --include=*.hpp \
  | grep -v cpp/parser/generated | grep -v cpp/ast/Expression
cpp/nfsim/NFcore2/legacy_bridge.cpp:112:            return functionExpression.evaluateWithFunctions(
cpp/nfsim/NFcore2/legacy_bridge.cpp:264:        const double result = parsed.evaluateWithFunctions(resolve, state.time(), resolveFunction);

I never profiled NFsim, so the coldness claim is unestablished there, not
disproven — and legacy_bridge.cpp:264 calls it once per rate-law evaluation
inside the NFcore2 driver, a different execution shape. That is a hypothesis
for whoever profiles that lane, not a finding.

Correctness gate (run on the candidate before rejecting it)

• ctest --test-dir build --output-on-failure -j4 → 100% tests passed out of 461, twice on the reverted tree. One earlier run reported 99%/461 with
  architecture_nfnext_cache failing on a filesystem error; it passed in
  isolation and in both later full runs. cpp/nfnext untouched here.
• Trajectory identity, both binaries from one tree at one commit: 4M-step
  ODE, 228,000,114-byte .gdat, sha256 54195c48...72f1 on both arms;
  .net sha256 f66e0457...8942 on both; cmp clean. Bit-identical.
• A first .gdat mismatch was sciMetabolic's MM fix, not mine — caught by
  rebuilding a true baseline at the same commit.

Reproduction

git show 6889fba:cpp/ast/Expression.cpp > /tmp/compilerbench/variants/BASE.cpp
for opt in O0 O1 O2 O3; do
  c++ -$opt -std=c++20 -DNDEBUG -I /tmp/compilerbench/variants -I <wt>/cpp \
      /tmp/compilerbench/exprbench.cpp /tmp/compilerbench/variants/BASE.cpp -o /tmp/compilerbench/mb_${opt}_A
  c++ -$opt -std=c++20 -DNDEBUG -I <wt>/cpp \
      /tmp/compilerbench/exprbench.cpp <wt>/cpp/ast/Expression.cpp -o /tmp/compilerbench/mb_${opt}_B
done
for rep in $(seq 1 15); do for v in A B; do ./mb_O2_$v 400000 | awk '/^TOTAL/{print "'"$v"'",$2}'; done; done
for rep in $(seq 1 6); do for v in A B; do /usr/bin/time -l ./mb_O2_$v 400000 2>&1 | awk '/instructions retired/{print "'"$v"'",$1}'; done; done

The candidate is mechanically reproducible with no semantic edit: prefix
text_.size() == len(lit) && to every text_ == "lit" between
case ExpressionKind::Unary and case ExpressionKind::ObservableRef (66
sites). Reconstructing it that way reproduced the measured file byte for byte.

The transferable lesson

Wall clock produced the false positive. The counter contradicted it — and that
contradiction is what settled the charge, not any resolved delta.

The step I skipped, now stated in the file: compare the between-arm delta
against the within-arm range of each arm before quoting any delta. Tightness
does not license it; a tight-looking counter makes you more confident, which
is when the check gets skipped. Mine had a 0.06–0.10% range and I still quoted
0.024% against it.

And the stronger property, which neither of my instruments had: a check that
needs no oracle at all because it contradicts an invariant rather than
another implementation — sciSignaling's frozen pool (kon=0, a conserved
pool at steady state is its own answer) is the model. Falsify cheap and
convention-free first; use a reference implementation only afterwards to
establish what the correct value is. Had the counter agreed with wall clock,
I would have shipped noise with two instruments pointing the same way and
neither able to say so.

Files

docs/PERF_EXPRESSION_CODEGEN_NO_WIN.md (new). No production file modified;
git diff 6889fba --stat shows only that one file. sciMetabolic's MM fix
(8dd441d) is deliberately not on this branch — it is an FP-result change
in the same evaluate path and is theirs to ship.

🤖 Generated with Claude Code