test(conformance) astubbs#242: one definition of correct for every client, anchored by Parallel Consumer itself by astubbs · Pull Request #387 · astubbs/parallel-consumer

astubbs · 2026-08-31T10:23:16Z

Part of #242.

depends on #386

Description

The behavioural oracle, made independently reviewable, with Parallel Consumer itself anchoring
it. One Java test module holds the scenarios every proxy client will be driven through and the
assertions about each, so that correct is defined once for every language rather than once per
language - and the engine is one of the rows it is checked against.

The claim this PR makes, and what to check it against. A scenario red against Parallel Consumer
driven by a plain Java function is a wrong scenario, not a broken client - there is nothing else
left for it to be. That only means something if the suite can say no, so every scenario was proven
able to fail by breaking the engine-side behaviour it pins, and the negative control was proven able
to fail in the direction that matters by two complementary arms. All of them are below, with the
cells each one reddened and the cells it left green.

Where this came from

Extracted from #293, the language-proxy god PR, as the rung above
#386 - whose "deliberately NOT in this rung" list names exactly this
content: the transport-parameterised scenario suite, its fixture interface, the per-transport
subclasses, and core as the control arm. Taken by path-scoped checkout rather than rewritten; every
place it differs is named below.

What runs, and what is deferred

Binding / cell	State	Why
`core`	runs, in every selection	Parallel Consumer driven by a plain Java function - the control arm
`java-direct` (shared matrix)	runs	The client API bound in-process to core, with no protocol underneath
`java-direct` (transport-parameterised suite)	runs	Same suite, driven through the client's own surface
`java-grpc` (both suites)	deferred	Needs a sidecar with a dispatch engine; the one on this stack answers every session `UNIMPLEMENTED`
the ten foreign clients	deferred	Their runners and the machinery that spawns them are the next extraction

Nothing deferred is stubbed. A stand-in engine would make agreement between bindings a statement
about the stand-in, which is the one thing this suite may never be.

The java-grpc decision, and why it is not a `@Disabled` test

Left for the engine rung, not brought disabled - because there is nothing to disable. A
@Disabled test still has to compile, and the classes the gRPC fixtures name do not exist on
any branch reachable from here: the harness's startEngine lane, ConfigureHandler,
ProxyProcessor. Bringing them would mean writing the engine, or writing a stub of it, and the
instruction not to weaken the scenarios rules both out. So the honest option was the one taken, and
the work is to make the absence visible rather than silent - which
docs/testing.md's rule ("a test that never runs is not a passing test, and
nothing goes red to tell you") is the whole reason to care about.

Three mechanisms, none of them a comment:

TheEngineArrivingMustBringTheGrpcBindingTest (conformance module) and
TheEngineArrivingMustBringTheGrpcCellTest (harness module) each assert the cell exists
exactly when the engine is reachable on that module's classpath. The equality runs both ways,
so it goes red whether the engine lands without the cell or the cell is written without an engine,
and it retires itself - nobody has to remember. Both were sabotaged, by pointing the engine
constant at a class that is on the classpath, and both failed naming what to do.
SelectorMatchingNothingFailsTest asserts that -Dpc.conformance.language=java-grpc fails.
Without that, a CI row naming the deferred cell would run the control arm alone and report green -
a java-grpc run that tested no client, indistinguishable from one that passed.
A deferred docs/inflight/ note carrying what the engine work has to do here, and the
reconciliation duty below.

A docs/test-hardening/ audit was considered and is deliberately not added. Those audits record
tests that exist and do not run; nothing here exists to be dark. Adding a dated audit for a test that
was never written would make the registry describe something it does not own, and the guards above
are stronger than an audit entry - an audit goes stale silently, and these go red.

Six sabotage arms, each restored byte-identically

Ten cells: two bindings by five scenarios. Each arm changes one term on the engine side - the
behaviour the scenario is a claim about, not the test - and the interesting half of each result is
which cells stayed green.

=== CONTROL: untouched tree ===
Tests run: 10, Failures: 0, Errors: 0                                        EXIT=0

=== ARM 1: the redelivery carries a reason the worker never gave ===
  RED: core/failed-redelivered, java-direct/failed-redelivered               EXIT=1
  green: the other eight

=== ARM 2: the worker pool runs wider than the ceiling the caller declared ===
  RED: core/in-flight-ceiling, java-direct/in-flight-ceiling                 EXIT=1
  green: the other eight

=== ARM 3: KEY ordering stops sharding by key ===
  RED: both ordering cells AND both ceiling cells                            EXIT=1
  green: the six that do not turn on shard identity

=== ARM 4: a completed record is never recorded as complete ===
  RED: eight cells
  GREEN: core/unreported-holds-back, java-direct/unreported-holds-back       EXIT=1

=== ARM 5: a record is counted complete the moment it is dispatched ===
  RED: the negative control, and eight others                               EXIT=1

=== ARM 6: the embedded-client lane seeds nothing (fixture wiring) ===
  RED: all five java-direct cells
  GREEN: all five core cells                                                 EXIT=1

=== RESTORED: control arm again, every file byte-identical ===
Tests run: 10, Failures: 0, Errors: 0                                        EXIT=0

Arms 4 and 5 are the pair that matters, and neither is worth much alone. Arm 4 breaks everything
the suite asserts about progress and the negative control stays green - which is only reassuring if
the negative control can go red, and arm 5 is that: count a record complete at dispatch and it is
the one that reddens. Together they say it is a control rather than a test that happens to pass.

Arm 6 is what says the two bindings are judged independently. Every cell is a separate engine
over separate mock clients, so a fixture defect in one binding must not be able to redden or rescue
the other. Seeding nothing into the embedded lane reddens exactly the five java-direct cells.

Arm 3 is the one whose blast radius is a finding rather than a nuisance. Breaking key sharding
also reddens the ceiling scenario, because that scenario seeds six distinct keys precisely so key
ordering is not what limits concurrency - collapse the shards and the instrument stops measuring what
it is named for. That is the scenario's own reasoning, confirmed from the outside.

The transport-parameterised suite was proven able to fail separately: dropping the outcome's produce
list on its way into core reddens the two tests that assert the response record - the full-path one
and its asynchronous twin - and leaves the other three green.

Two experiments that were not sabotage

The mock-consumer ordering fix has a control arm and one term changed. The first commit reorders
two shared core fixtures to record beginning offsets before assigning partitions. The baseline is
green either way, so the race was widened rather than raced - a 3s sleep between the two calls, with
only their ORDER differing:

seed then assign, 3s window  ->  DirectSpikeConformanceTest  5/5 PASS
assign then seed, 3s window  ->  DirectSpikeConformanceTest  5/5 ERROR
                                 45 occurrences of "MockConsumer didn't have beginning offset"

A full-suite failure was attributed rather than assumed. One core run failed
ParallelEoSStreamProcessorTest.executorThreadsInterruptedOnShutdownTimeout, which
docs/inflight/bug-857-family.md already names as a load flake -
"two other timing-sensitive tests each failed exactly once across the same session's ~10 heavy runs".
Rather than accept that, the ledger's own prescribed method was run: the same command with and
without the change. The full core suite is green twice with the change and once without; the test
alone is green 3/3. One failure in four full-suite runs, on a box also running a concurrent
build - reported as a rate rather than a verdict. The mechanism does not connect either: the
reorder's failure mode is a poll-thread kill naming a missing beginning offset, not an
interrupted-executor assertion.

Three things this PR found

Error Prone cannot read Jabel-desugared records, and it crashes rather than reporting. Jabel
requires @Desugar on every record it sees - including at release 17, where the source branch's
comment claims it is a no-op - and then rewrites the record into a class whose generated members
carry no source positions. Error Prone 2.42.0 dies on that in UnnecessaryStringBuilder
(invalid replacement: [0, -1)) and, once that check is suppressed, again in DuplicateBranches. A
crash is not a finding: it fails the whole compilation, attributed to line 1 of whichever file javac
listed first, which is why it reads as unrelated to the file that caused it. Neither term can
move - the root pom pins Error Prone at 2.42.0 because 2.43.0 needs a JVM this build cannot use,
and Jabel serves the release 8 target. The five value types are plain final classes instead, which is
what the rest of the repository already does: grep -rn "record " over master finds no Java record
in any module, and this PR's own SpikeFixture.Seed was already written that way. The reasoning is
recorded at the site rather than in this body alone. The source branch could not have met this -
its root pom has no Error Prone at all, which is the same shape as the EveryModuleWiresUpArchUnitTest
surprise #386 hit one rung below.

The harness's explicit onPartitionsAssigned after rebalance is redundant on kafka-clients
3.7+, which fires the registered listener from inside rebalance. Found by sabotaging it and
watching nothing go red - a null result worth reporting, because it means the inherited comment
saying rebalance "fires no listener" is stale. Left in place: it is defensive, the sibling
fixture in the direct module does the same, and deleting inherited belt-and-braces on the strength of
one library version is not this rung's call.

A parent rung's pom claimed work this rung would do, and could not. The harness module's pom said
the engine-backed conformance run "arrives with the conformance rung stacked above this one". It does
not - that is the java-grpc decision above - so the sentence is corrected rather than left to rot,
and it now names the engine rung and the guard test that holds it.

Where the harness ended up, and the one reconciliation this leaves

ConformanceHarness is feats/proxy-requirements's ProxyHarness with the engine lane removed. It
is renamed and relocated, and both halves are deliberate: without the engine lane it has nothing
to do with the proxy, and the conformance suite is its only consumer here. That also gives the rung a
sharper claim than it would otherwise have - the conformance module does not depend on the sidecar
at all, which its pom records as a reactor measurement rather than an assertion.

They are one class, and whoever lands the engine reconciles them rather than keeping both. That
duty is written in the class's own header, in the pom, and in the inflight note, because it is exactly
the kind of thing a merge resolves silently.

Four other places this differs from #293 on purpose

The foreign-language machinery is not taken - the runner registry, the process driver's spawn
half, the sidecar shim, the repo-layout helper and the three tests that exercise them. ConformanceBindings
keeps its two-registry concatenation so A9 is purely additive there. What is taken is everything
language-blind, including the runner contract's constants and the stdout observation format, because
the in-process bindings are held to exactly that contract.
SelectorMatchingNothingFailsTest is re-pointed at bindings that exist here, and gains the
deferred-cell case described above. Asserting a message names python when nothing registers
python would be a test about a string.
PrescribedRun accumulates diagnostics as lines rather than into a StringBuilder, and builds
its stdout from DispatchObservation's own toString rather than restating the line format. Two
spellings of one contract is what this suite exists to prevent.
The module's src/test/CLAUDE.md write-time bridge is left behind, for the same reason both
rungs below left theirs: the document it imports arrives with docs(testing): make prove-it-red arrive when a test is written, not when it is looked up #378.

How it was verified

JDK 17, macOS, on this branch:

The conformance module's own suite green: the matrix's ten cells, the harness's own five (including
its negative control, which proves the fixture can fail), the transcript's overlap computation, the
selector's six cases, the scenario-coverage checks, the deferral guard, and the ArchUnit wrapper.
The transport-parameterised suite green under the direct transport, in the direct module.
The full core suite, twice, green (and once more with the fixture change reverted - see the
attribution above).
bin/check-all.sh on its real exit code: 0. Two gates report CANNOT RUN -
check-proto-breaking.sh and check-proto-lint.sh, both wanting a buf this machine does not
have. Both belong to a base rung, neither is affected by this diff, and CI is where they run.
COPYRIGHT_CHECK_REQUIRE_FORK_POINT=1 bin/check-copyright-headers.sh - 0 violations against a
real fork point.
bin/check-docs-data.sh green with this module's maturity and evidence rows present.

Commits were made with --no-verify, and the reason is recorded rather than assumed. The
pre-commit hook resolved the wrong project directory and gated this branch against a different
worktree's tree - it reported dangling citations in bench/README.md, a file that does not exist on
this branch at all. bin/check-file-refs.sh run in this worktree reports zero, and
bin/check-all.sh exits 0. The harness fix is in flight on #382.

Known-red, and not a fault

Check PR Dependencies is red until feat(client) astubbs#242: the Java reference client - one surface, two transports, and a handshake against a real sidecar #386 and its own parents
merge. That is the dependency gate doing its job on a stack rung.
claude-review is red until somebody asks for a review; every rung below is in the same state.

roadmap-stage: N/A - the language-proxy-sidecar entry names #293 as its carrier and sits at limited-poc on the strength of that PR's content. Landing a test module on master makes no more advanced artifact exist - there is still no engine for a client to talk to - so its stage is unchanged.

Checklist

Docs updated - the module README.md (the runner contract and what is deferred), AGENTS.md's
module table, the module-maturity and testing-evidence records for the new module, a correction to
the java-client evidence row whose limitation said the conformance suite was not on this branch,
the harness module's pom comment, and a deferred docs/inflight/ note for the two absent cells.
User-facing feature documentation data added under docs/features/ - N/A - nothing here is a
user-facing feature. The module has no main sources, publishes to no registry, and cannot be
depended on; the release-documentation records it earns are the maturity and evidence rows, which
are in this PR and say in both directions what is and is not evidenced.
Tests added/updated - this PR is tests. Six sabotage arms with control arms either side, plus
two guard tests each proven able to fail, above.
Title & body reflect the final content of this PR
Ran ce-simplify and ce-code-review locally - N/A - asked for @claude review this on the PR
instead. Most of the diff is code taken from feat(proxy) astubbs#242: a sidecar giving non-JVM runtimes key-ordered concurrency, in eleven languages #293 where it was already
reviewed, and the changes this rung made to it are named one by one above.

MockConsumer#rebalance assigns the partitions and, since kafka-clients 3.7, fires the registered rebalance listener from inside the call - so Parallel Consumer's broker-poll thread can be polling those partitions before the method returns. A poll of an assigned partition whose beginning offset has not been recorded throws IllegalStateException: MockConsumer didn't have beginning offset specified, but tried to seek to beginning which kills the poll thread and takes the engine down with it. The test then fails on whatever deadline it was awaiting, seconds later, naming a timeout rather than the cause. Both shared fixtures recorded the offsets AFTER the assignment, leaving a window: LongPollingMockConsumer#subscribeWithRebalanceAndAssignment, and MockConsumerTestBase, where the window was two calls wide. Seeding first REMOVES the window rather than narrowing it. Read off the kafka-clients bytecode rather than inferred: updateBeginningOffsets is a map merge that needs no assignment, so it can always be called first; rebalance never touches that map, so seeding first is never undone; and resetOffsetPosition throws exactly when the map has no entry for the partition under EARLIEST. There is no interleaving in which the old order was correct. The experiment, with a control arm and one term changed. The baseline is green either way, so the race was widened rather than raced: a 3s sleep between the two calls, and only their ORDER differing. seed then assign, 3s window -> DirectSpikeConformanceTest 5/5 PASS assign then seed, 3s window -> DirectSpikeConformanceTest 5/5 ERROR, 45 occurrences of the exception above The write-up this commit brings records the original measurement, made against the gRPC sibling of that test on feats/proxy-requirements, plus the sweep of every other call site of rebalance/assign near updateBeginningOffsets in the tree - two needed fixing, and they are the two changed here. Extracted from #293.

@test

…rect transport The transport-parameterised suite: the vertical slice a client has to carry - a record seeded, dispatched, handed to the user's processor, its outcome applied, its produce payload produced, its offset committed - written once and run against a transport it cannot name. SpikeConformanceTest and SpikeFixture live in the api module's test-jar, which is dependency-free on core, protobuf and gRPC. That is what makes the abstraction checkable rather than asserted: a transport type reaching SpikeFixture fails the direct module's bannedDependencies build, so the suite cannot quietly learn which transport it is running against. Each transport module supplies a subclass whose entire content is a fixture, and every @test runs under it with no transport-specific branch - so a behavioural difference between two runs is the transport's bug, never the suite's ambiguity. DirectSpikeConformanceTest is the first cell: core bound in-process through its own mock Kafka clients, no broker, no Docker, in the surefire lane. The gRPC cell is NOT here, and the reason is not that it was forgotten. Its fixture needs the sidecar's dispatch engine, and the sidecar on this stack hosts none - it answers every session UNIMPLEMENTED. It is absent rather than disabled because a @disabled test still has to COMPILE, and the classes its fixture names exist on no branch reachable from here. What guards the absence is TheEngineArrivingMustBringTheGrpcCellTest, in the module that already depends on the sidecar: it asserts the cell exists EXACTLY when the engine is on the classpath, so it goes red whether the engine lands without the cell or the cell is written without an engine. Sabotage arm: point its engine constant at a class that is on the classpath, and it fails naming what to do. Proven able to fail: dropping the outcome's produce list on its way into core reddens the two tests that assert the response record - the full-path one and its asynchronous twin - and leaves the other three green. The fixture is proven load-bearing too, by the widened-window experiment in the commit below. The harness module's pom said this rung would land the engine-backed conformance run. It cannot, so that sentence is corrected rather than left to rot. Extracted from #293.

@desugar

…r itself as the control arm parallel-consumer-proxy-conformance: one Java test module holding the scenarios every client will be driven through and the assertions about each, so that "correct" is defined once for every language rather than once per language. Why one definition rather than eleven. If each client asserted for itself, two of them disagreeing would leave nobody able to say which was wrong - written once, agreement between clients is evidence; written eleven times, it is coincidence. So a client asserts nothing: it does what a closed set of behaviour tokens prescribes, records two observations per record, and exits with a status. The suite decides what any of it means. CORE IS ONE OF THE BINDINGS, AND THAT IS THE POINT OF THE RUNG. Every other binding puts a client - and eventually a protocol and a language runtime - between the scenario and the engine, so a red run has several suspects and the client is always the first one looked at. A scenario red against a plain Java function is a WRONG SCENARIO; there is nothing else left for it to be. It runs in every selection and cannot be selected away, because that answer is worth nothing if it arrives in a different CI job hours later. It earned its place on the day it was written: the redelivery scenario went red against core because the binding read the failure reason off the Throwable core records - the wrapper's message rather than the user's - which would have read as "the client mangles the reason" in whatever language it first appeared in. The prescription is not relaxed for an in-process binding. PrescribedRun is the runner contract carried out in this JVM, and the control arm and the client binding execute the same instance of it, so no assertion can be written to suit either. WHAT RUNS: core and java-direct, five scenarios each. java-grpc is deferred with the engine it needs, absent rather than stubbed - a stand-in would make agreement between bindings a statement about the stand-in. The ten foreign runners and the machinery that spawns them are the next extraction. Both deferrals are guarded, not noted: TheEngineArrivingMustBringTheGrpcBindingTest asserts the binding is registered exactly when the engine is reachable, and SelectorMatchingNothingFailsTest asserts that naming the deferred cell FAILS rather than running the control arm alone and reading as a java-grpc pass. EVERY SCENARIO PROVEN ABLE TO FAIL, by breaking the engine-side behaviour it pins and restoring byte-identically. Same-magnitude-different-position, not a bigger hammer: a redelivery carrying a reason the worker never gave -> the 2 redelivery cells a worker pool wider than the declared ceiling -> the 2 ceiling cells key ordering that stops sharding by key -> the 2 ordering + 2 ceiling cells a success never recorded -> 8 cells; the negative control's 2 STAY GREEN a record counted complete at dispatch -> the negative control goes red the embedded lane seeds nothing -> all 5 java-direct cells; all 5 core cells green The last three are the ones worth reading. The two success-bookkeeping arms are complementary and it takes both to say the negative control is a control: one makes everything else red while it stays green, the other makes it alone go red. The seeding arm is what says the two bindings are judged independently and that the fixture wiring is load-bearing rather than decorative. Two things this found that the source branch could not have met, because master gained both after it was cut: - ERROR PRONE CANNOT READ JABEL-DESUGARED RECORDS, and it crashes rather than reporting. Jabel requires @desugar on every record it sees - even at release 17 - and then rewrites the record into a class whose generated members have no source positions; Error Prone 2.42.0 dies on that in UnnecessaryStringBuilder ("invalid replacement: [0, -1)") and, with that one suppressed, again in DuplicateBranches. Neither term can move: the root pom pins 2.42.0 because 2.43.0 needs a JVM this build cannot use, and Jabel serves the release 8 target. The five value types are plain final classes instead, which is what `grep -rn "record "` says the rest of the repository already does - master has no Java record in any module. - The harness's explicit onPartitionsAssigned after rebalance is REDUNDANT on kafka-clients 3.7+, which fires the registered listener from inside rebalance. Found by sabotaging it and watching nothing go red. Left in place - it is defensive, matches the sibling fixture, and removing inherited belt-and-braces on the strength of one library version is not this rung's call - but recorded so the next reader does not rediscover it. Extracted from #293, which keeps the engine the deferred cells wait on.

github-actions · 2026-08-31T10:23:28Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details
maven/bz.stub.parallelconsumer:parallel-consumer-proxy-client-java-api	0.6.0.0-SNAPSHOT	Unknown	Unknown
maven/bz.stub.parallelconsumer:parallel-consumer-core	0.6.0.0-SNAPSHOT	Unknown	Unknown
maven/bz.stub.parallelconsumer:parallel-consumer-proxy-client-java-direct	0.6.0.0-SNAPSHOT	Unknown	Unknown
maven/org.apache.commons:commons-lang3		Unknown	Unknown
maven/org.junit.jupiter:junit-jupiter-params		Unknown	Unknown

Scanned Files

parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-direct/pom.xml
parallel-consumer-proxy-clients/parallel-consumer-proxy-conformance/pom.xml

github-actions · 2026-08-31T10:24:11Z

✅ Duplicate Code Report

Two engines run in parallel for cross-validation. Each has its own thresholds tuned to its baseline - the real safety net is the per-engine "max increase vs base" check.

✅ PMD CPD

	PR	Base	Change
Clones	34	32	🫤 +2
Duplicated lines	1061	1017	:face_with_raised_eyebrow: +44
Duplication	0.13%	0.13%	🫤 +0.01%

Rule	Limit	Status
Max duplication	0.5%	✅ Pass (0.13%)
Max increase vs base	+0.1%	✅ Pass (+0.01%)

⚠️ 2 new clones introduced

16 lines (82 tokens): parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:90 <-> parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:192
6 lines (71 tokens): parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:78 <-> parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:107

✅ jscpd (language-agnostic)

	PR	Base	Change
Clones	100	97	🫤 +3
Duplicated lines	1421	1385	:face_with_raised_eyebrow: +36
Duplication	1.01%	1.02%	🙂 -0.01%

Rule	Limit	Status
Max duplication	2%	✅ Pass (1.01%)
Max increase vs base	+0.1%	✅ Pass (-0.01%)

⚠️ 3 new clones introduced

9 lines: parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-direct/src/test/java/bz/stub/parallelconsumer/client/direct/DirectParallelConsumerClientTest.java:9 <-> parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-direct/src/test/java/bz/stub/parallelconsumer/client/direct/DirectSpikeConformanceTest.java:10
7 lines: parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:106 <-> parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:77
23 lines: parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:192 <-> parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:90

_{Powered by astubbs/duplicate-code-cross-check}

github-actions · 2026-08-31T10:24:13Z

+            awaitCommittedOffset(fixture, 1);
+
+            assertWithMessage("the user function ran exactly once")
+                    .that(invocations).containsExactly("hello");


⚠️ Duplicate code detected - 16 lines duplicated with parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:192

github-actions · 2026-08-31T10:24:14Z

+    @Test
+    void oneRecordTravelsTheFullPathAndItsResponseIsProduced() {
+        var topic = "spike-one-record-full-path";
+        var invocations = new CopyOnWriteArrayList<String>();


⚠️ Duplicate code detected - 6 lines duplicated with parallel-consumer-proxy-clients/parallel-consumer-proxy-client-java/parallel-consumer-proxy-client-java-api/src/test/java/bz/stub/parallelconsumer/client/conformance/SpikeConformanceTest.java:107

github-actions · 2026-08-31T10:24:34Z

🧪🔒 Quarantine Lane Report

Quarantined test	Outcome	Owner	Meaning
`ProducerManagerTest.producedRecordsCantBeInTransactionWithoutItsOffsetDirect`	🔴 failing (expected)	#262	quarantine holding

_{🔴 expected while the owner PR is open · 🟡🎲 flapper, pass proves nothing · 🚨 a deterministic quarantined test passing means its fix landed: delete its @Quarantined annotation + docs/quarantined-tests.md entry (a merge-blocking review thread has been opened). Lane: non-gating; rules: see the Quarantine Audit check.}

github-actions · 2026-08-31T10:33:02Z

⚠️ SpotBugs Report

431 bug(s) found (rule-level exclusions only - see docs/inflight/static-spotbugs-rule-registry.md). See the annotations on the Files Changed tab for details.

…s engine back Merges `feats/proxy-dispatch-clients` (#390, the top of the Wagon A stack) into this branch, so that #293 can be retargeted onto that rung and its displayed diff collapse to the engine residue. This is the plan's "retarget move" - `docs/plans/2026-08-31-001-process-god-branch-decomposition-plan.md` - and it is an ordinary merge: no history was rewritten and nothing was force-pushed. The pre-merge tip is preserved as `origin/backup/pre-stack-merge-293`. The merge also brings `master` forward, because the stack is current with it and this branch was not. That is most of the volume and most of the conflicts. ## Which side won, and why The rule was: the god branch's SEMANTIC content, expressed on the stack's refined STRUCTURE. - **Engine-only files stayed** - `ProxyProcessor`, `ConfigureHandler`, the dispatch waves, epochs, leases, reconnect and the produce path. They are the residue this campaign exists to expose. - **Stack-only files were taken** - the ten foreign clients, the runner registry, the sidecar shim, the self-retiring guards, `Main` and `ParentDeathWatchdog`. - **Records lost, everywhere.** The stack converted five value types from Java records to plain final classes because Jabel rewrites a record into a class with no source positions and Error Prone 2.42.0 crashes rather than reporting on it. This branch never met that only because its root pom had no Error Prone; master's does now, so the conversion is load-bearing here. - **Dependency pins are the stack's**: protobuf 3.25.8 (`requireUpperBoundDeps` refuses 3.25.5 against grpc-protobuf's transitive ask), gson 2.14.0, `error_prone_annotations` 2.48.0 in the proxy and the Java client aggregator and 2.47.0 in the protocol module - deliberately unequal, because their highest requesters differ. - **`clients.yml` is this branch's**, not the stack's. The stack's copy is a smaller earlier file whose own header says it has no conformance lane, no per-language static analysis and no dependency audit; only its `--component clippy` fix and its workflow-level `PC_FOREIGN_CLIENTS_STRICT` were folded in. - **Master's copy won for everything master owns** - the gate scripts, the copyright checker and its self-test, `repo-hygiene.yml`, `docs/ci.md`, `docs/copyright.md`. This branch's only delta in most of those was a copyright header in a pre-canonical form, written before #338 landed the canonical one on master. - **The two ledgers were merged as unions**, not resolved to a side. `docs/inflight/bug-857-family.md` keeps both sides' sightings, this branch's renumbered per the file's own stated convention; `docs/quarantined-tests.md` drops three entries whose owning PRs retired the annotation on master, because a registry entry without a `@Quarantined` test is a hard gate failure. ## The verdict-free work return, re-expressed on master's shape Unit U4 is still on this branch as `WorkManager.onAbandonedResult` and its call sites, and `WorkManager.java` auto-merged keeping them - so resolving `WorkContainer`, `ProcessingShard` and `ShardManager` to master alone would have compiled to nothing. Master moved underneath it: #335 collapsed `inFlight` and `maybeUserFunctionSucceeded` into one atomic `ExecutionState`, and #336 and #373 replaced the shard's raw `availableWorkContainerCnt` with a compare-and-set claim. `ProcessingShard.onAbandoned` and `ShardManager.onAbandoned` are therefore taken verbatim from `feats/proxy-verdict-free-return` (#295), which is cut on current master and is the authoritative re-expression - it routes the restore through `includeInSelection` rather than incrementing a counter that no longer exists. `WorkContainer` is NOT #295's, and the difference is deliberate: that branch marks abandonment with an `AtomicBoolean` cleared by the claim winner, while this branch keys it by delivery (`markAbandoned(long)`, `isAbandonedForCurrentDelivery`, `isReturnForSupersededDelivery`). The delivery-keyed form is strictly more: it tells a late return for delivery n from a live return for delivery n+1, which the boolean cannot, and acting on that confusion ends a running flight and decrements `numberRecordsOutForProcessing` twice. It also needs no clearing on redelivery, so it does not race the claim. Master's own `deliveryCount` - already an `AtomicLong` incremented only by a won claim - supplies the identity, so the field is the one thing added. ## The reconciliation duties #387 and #390 recorded Both are discharged, and the guards they left behind now pass because the gap closed rather than because anything was edited. **`ProxyHarness` and `ConformanceHarness` are one class again.** #387 cut the latter out of the former with the engine lane removed, and wrote into the class header, the module pom and a deferred inflight note that whoever landed the engine must merge them - leaving open which module it ends up in, on the grounds that "the engine lane decides". It decides the sidecar module: the lane needs `ConfigureHandler` and `ProxyProcessor`, and `TestModeMain` - the process a spawned foreign runner actually starts - is in that same test tree and would otherwise have to import the conformance module and close a cycle. So the NAME is the extracted rung's and the MODULE is this branch's, and `parallel-consumer-proxy-conformance` reaches it through the proxy test-jar its pom already declared. The stack's refinements to the class survive: `Delivery` stays a plain final class, and the `startEmbeddedClient` lane stays beside the restored `startEngine`. **`ConformanceDriver.spawnAgainst` is back**, with `LanguageRunner` a `ConformanceBinding` again - it stopped being one on #390 for exactly as long as there was no engine to spawn against - and the ten foreign bindings are registered in `ConformanceBindings.selectable()`. `ConformanceBindings.deferredUntilTheEngineArrives()` is kept and now returns an empty list, which is the honest reading: nothing is deferred. It is not deleted because `TheEngineArrivingMustBringTheForeignCellsTest` reads it, and because the next capability to arrive without a cell needs that list rather than a new one. ## Two duplications the merge surfaced, resolved rather than left - **The protocol documents existed twice.** #383 moved `protocol-specification.md` and `client-authoring-guide.md` into `parallel-consumer-proxy-protocol/docs/` and predicted the rename conflict here. The protocol module wins on the merits - all three artifacts are the contract rather than the server - but the CONTENT is this branch's, because the stack's copies carry "no client generates from this schema yet" and a `git show` fallback for a Go generator script that exists here. `SpecificationCoverageTest` moved with them; every citation was re-pointed. - **`HarnessScenario` existed twice**, once per harness. The stack's plain-class version won, in the sidecar module beside the class it serves. ## Five Java records had to become plain final classes, or nothing compiled `master` gained Error Prone after this branch forked, and Error Prone 2.42.0 does not report on a Jabel-desugared record - it crashes, failing the whole compilation with a message attributed to line 1 of an unrelated file. #387 hit this one rung down and converted its five value types; the same thing was waiting here for `ProxyProcessor.ManifestOutcome`, `ManifestReconciler.Reconciliation`, `InFlightRegistry.InFlight`, `LivenessSettings` and `TestModeMainTest.Run`. Neither term can move - the root pom pins Error Prone at 2.42.0 because 2.43.0 needs a JVM this build cannot use, and Jabel serves the release 8 target - so the records lost, which is what the rest of the repository already does. `grep -rn "record "` now finds none. ## Two defects this merge introduced and then fixed - **The default lane ran a test that needs a profile.** Keeping BOTH sidecar-spawning tests - this branch's engine-backed `OneRecordThroughTheSidecarTest` and the stack's engine-less `SidecarHandshakeTest` - left the Kotlin and Scala exclusion property naming only the first, so the second ran in an ordinary build with no `sidecar-classpath.txt` and failed. Both are now behind one regex, which is also the arrangement that makes the pairing legible. - **Every proxy module's data record existed twice.** This branch keeps module-maturity and testing-evidence records in per-module `docs/data/*.d/` fragments; the stack, which never imported that mechanism, wrote them into the monolithic files. The merge kept both and `bin/check-docs-data.sh` went from green to 37 structural problems. The fragments win, and deleting the monolithic copies also deleted eleven rows asserting "NO RECORD HAS CROSSED THE WIRE" - true on a stack whose sidecar hosts no engine, false here. ## How it was verified JDK 17, macOS. Full default reactor `./mvnw --fail-at-end test`: **BUILD SUCCESS**, every module green including the conformance suite - so the `java-grpc` cell runs against a live engine for the first time, and `GrpcSpikeConformanceTest` answers all five scenarios over a real gRPC stream. The four self-retiring guards #387 and #390 left behind all pass, and **none of them was edited**: `TheEngineArrivingMustBringTheGrpcBindingTest`, `TheEngineArrivingMustBringTheGrpcCellTest`, `TheEngineArrivingMustBringTheForeignCellsTest` (which asserts about every registered language, not one) and `SelectorMatchingNothingFailsTest`. `bin/check-all.sh` real exit code 1, with three gates failing and all three inherited, established against the pre-merge tip rather than assumed: `check-inflight-tags` reports the same 53 problems before and after; `check-file-refs` was already red for the `@`-prefixed bridge imports that #378 fixes; and `check-branch-self-reference` did not exist on this branch before the merge - its findings are on notes inherited from `master` byte-identical. `check-proto-lint` and `check-proto-breaking` report CANNOT RUN for want of `buf`. **Not run locally: `-Dpc.foreignClients`.** It builds C++ and Swift inside containers, and this box is under a low-disk warning that names a container build as what would tip it over. CI's per-language rows are where that lane runs. ## Two findings recorded rather than fixed - `ParallelEoSStreamProcessorTest.processInKeyOrder` failed its own preamble sanity check once here - and the control arm reddened `master` HARDER: unmodified `master`, uncontended, failed all three parameterisations where this branch under concurrent load failed one. That refutes the existing ledger's "the base branch was green" row, and is the first evidence the flake is master-state. Recorded in `docs/inflight/test-processinkeyorder-sanity-check-races-the-first-poll.md`. - One `java-grpc` conformance run left its last record unsettled, 1 of 2 full-reactor runs and green 3 of 3 in isolation. `core` and `java-direct` were green in the same run, so the suspect is the engine's settle path under a full ceiling rather than the scenario. The scenario was NOT weakened - `docs/inflight/proxy-a-java-grpc-ceiling-run-left-its-last-record-unsettled.md`. ## What is deliberately NOT done here `Main#sessionServiceFactory` still returns `NoEngineSessionService`, so the production entry point hosts no engine. That is unit U10's, which #384 named as still having no PR, and it owns the drain that must land with the engine - plus eight cross-language handshake tests assert the `UNIMPLEMENTED` refusal by status code specifically. Recorded rather than faked, in `docs/inflight/proxy-the-production-entry-point-still-hosts-no-engine.md`. The engine is exercised end to end through `TestModeMain` regardless.

astubbs added 3 commits August 31, 2026 20:42

astubbs mentioned this pull request Aug 31, 2026

feat(proxy) astubbs#242: a sidecar giving non-JVM runtimes key-ordered concurrency, in eleven languages #293

Draft

6 tasks done

github-actions Bot reviewed Aug 31, 2026

View reviewed changes

astubbs marked this pull request as draft August 31, 2026 12:36

astubbs mentioned this pull request Aug 31, 2026

feat(clients) astubbs#242: ten foreign dispatch clients, and the runner registry that will drive them #390

Draft

5 tasks done

Navigation Menu

Repository navigation

test(conformance) astubbs#242: one definition of correct for every client, anchored by Parallel Consumer itself - #387

test(conformance) astubbs#242: one definition of correct for every client, anchored by Parallel Consumer itself#387
astubbs wants to merge 3 commits into
feats/proxy-java-clientfrom
feats/proxy-conformance

astubbs commented Aug 31, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

github-actions Bot Aug 31, 2026

Uh oh!

github-actions Bot Aug 31, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

github-actions Bot commented Aug 31, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

astubbs commented Aug 31, 2026

Description

Where this came from

What runs, and what is deferred

The java-grpc decision, and why it is not a @Disabled test

Six sabotage arms, each restored byte-identically

Two experiments that were not sabotage

Three things this PR found

Where the harness ended up, and the one reconciliation this leaves

Four other places this differs from #293 on purpose

How it was verified

Known-red, and not a fault

Checklist

Uh oh!

github-actions Bot commented Aug 31, 2026

Dependency Review

OpenSSF Scorecard

Scanned Files

Uh oh!

github-actions Bot commented Aug 31, 2026

✅ Duplicate Code Report

✅ PMD CPD

✅ jscpd (language-agnostic)

Uh oh!

github-actions Bot Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 31, 2026

🧪🔒 Quarantine Lane Report

Uh oh!

github-actions Bot commented Aug 31, 2026

⚠️ SpotBugs Report

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

The java-grpc decision, and why it is not a `@Disabled` test