Repository navigation
Conversation
Ten Maven modules had none. Each README was written from the module's own source - the pom, the main classes, the tests - not from its directory name, and follows the shape the two proxy READMEs set: what the module is, what is in it, the two or three things to know before reading the code, then who owns the topic. The proxy modules' experimental banner is deliberately absent: it belongs to unreleased work, and these nine are shipped. What reading each module established: - core: the parts a newcomer meets first are the controller/poller split (AbstractParallelEoSStreamProcessor vs BrokerPollSystem, sharing WorkManager state across the boundary), state/ (WorkManager over ShardManager, plus a purpose-built RetryQueue whose uniqueness key differs from its sort key), and offsets/ (every candidate encoder run simultaneously, packSmallest wins). Two facts worth writing down because nothing else states them together: ordering is only a shard-assignment decision - UNORDERED shards identically to PARTITION, and what differs is ProcessingShard.isOrderRestricted - and the KafkaStreams OffsetEncoding values exist solely to recognise metadata Kafka Streams wrote into the same field and reject it via KafkaStreamsEncodingNotSupported. - vertx, reactor, mutiny: all three are thin. Each is an ExternalEngine subclass, and the three constraints that matter are core's, not theirs - ExternalEngine rejects transactional commit mode outright (so none of the three offers exactly-once), pins the worker pool to one dispatch thread, and defers success to the async completion rather than the function's return. Written once in core's README and cross-referenced from the other three. Mutiny's Java 17 floor is cited to its pom's release.target comment and to README.adoc's java-version-per-module anchor rather than re-explained. - examples: none of the five has a main method. They are compiled, tag-delimited source that src/docs/README_TEMPLATE.adoc include::s, which is why CoreApp keeps methods that are never called, and they are driven by their own tests. Each child README says which test runs it and whether it needs Docker - example-metrics and example-streams do, because failsafe selects on the package name integrationTest*, not on the source root, so they run under failsafe from src/test/java. Where drift is most likely, and what was done about it: - Restating the root README.adoc. It owns ordering, retries, commit modes, shutdown, metrics and the module-choice tables; every module README points at a named anchor there instead of summarising. Anchors were grepped before being committed, per the cite-by-anchor rule. - Restating docs/features/. That directory owns the per-capability records and most of them carry module: parallel-consumer-core, so it is routed to by name rather than paraphrased. - Pom comments. Four of these modules carry long, load-bearing pom comments (mutiny's release.target, core's junit parallelism property, example-metrics' micrometer pin, example-streams' jackson floor). Each README names the comment and refuses to duplicate its reasoning - the pom is the owner, and a copy here would be the first thing to go stale. Two follow-ups the owner may want, neither taken here: README.adoc's module-maturity table calls the examples "runnable examples", which no main method backs up; and the examples' integration tests sit in src/test/java rather than src/test-integration/java, which AGENTS.md's one-line suite summary does not cover. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SaBMWMJGbbpHQybuZUPCwa
…ignment from order restriction The README said ordering is a shard-assignment decision 'and nothing more', then immediately described a second thing that differs. Owner's correction, 2026-08-15: UNORDERED's relationship to partition is not that simple. Both halves of the original claim are true in the code - ShardKey.of maps PARTITION and UNORDERED to the same topic-partition key, and isOrderRestricted is false only for UNORDERED - but presenting the second as a footnote to the first invites the reading that UNORDERED is PARTITION under another name. It is not, and the difference is the one users actually feel. Now stated as two independent axes with the ceilings tabulated, because that is where the counterintuitive part lives: under UNORDERED the shard count still equals the partition count while concurrency is bounded by the executor pool instead, so shards are the unit of accounting rather than of parallelism. PARTITION is the mode whose ceiling really is the partition count. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
✅ Duplicate Code ReportTwo engines run in parallel for cross-validation. Each has its own thresholds tuned to its baseline - the real safety net is the per-engine "max increase vs base" check. ✅ PMD CPD
No new clones introduced by this PR. ✅ jscpd (language-agnostic)
No new clones introduced by this PR. Powered by astubbs/duplicate-code-cross-check |
✅ SpotBugs ReportNo bugs found (new bugs only — baseline from base branch excluded). |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 97f1a5357e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| A second surprise on the first axis: key-ordered shards key on the **topic name**, not the | ||
| topic-partition, so the same key arriving on two assigned partitions still runs in one shard. |
There was a problem hiding this comment.
When the same key appears on two assigned partitions, this guarantee is false: ShardKey.KeyOrderedKey retains a TopicPartition constructed from both rec.topic() and rec.partition(), and Lombok includes that field in equality, so the records enter different shards and may run concurrently. This matters with custom partitioners or historical repartitioning; document the implemented topic-partition-plus-key scope rather than promising cross-partition serialization.
AGENTS.md reference: AGENTS.md:L42-L46
Useful? React with 👍 / 👎.
| An adapter that lets your record handler return a SmallRye Mutiny `Uni` (or a `Multi`), completing | ||
| and committing the record only when that asynchronous work resolves. Parallel Consumer still owns |
There was a problem hiding this comment.
A caller following this sentence cannot return a Multi directly: the only entry point accepts Function<PollContext<K,V>, Uni<T>>, so a lambda returning Multi fails to compile. The implementation only recognizes a Multi emitted as the item of the returned Uni; describe that nested case explicitly or document only Uni as the supported handler return type.
Useful? React with 👍 / 👎.
| - **What gets committed is the highest *sequentially* succeeded offset; everything completed above | ||
| it lives in the commit metadata.** That is the whole reason `offsets/` exists, and it is bounded - |
There was a problem hiding this comment.
When users inspect consumer-group state, Kafka does not store the highest sequentially succeeded record offset itself: PartitionState.createOffsetAndMetadata() uses getOffsetToCommit(), which returns getOffsetHighestSequentialSucceeded() + 1. This off-by-one description can mislead recovery and commit diagnostics; say that the commit advances to the next offset after the contiguous succeeded range, or defer entirely to the identified owner.
AGENTS.md reference: AGENTS.md:L42-L46
Useful? React with 👍 / 👎.
| - The options block is unchanged from the core example - `ordering(KEY)`, a consumer, a producer. | ||
| **Switching engines is a change of processor type, not of configuration**: `new | ||
| ReactorProcessor<>(options)` in place of `ParallelStreamProcessor.createEosStreamProcessor`. |
There was a problem hiding this comment.
A user switching from the core example does not retain the same concurrency configuration: CoreApp.setupParallelConsumer() explicitly sets maxConcurrency(1000), while both ReactorApp and VertxApp omit it and therefore inherit DEFAULT_MAX_CONCURRENCY of 16. The claim that only the processor type changes hides a substantial throughput and in-flight-work change; either carry the setting across or call out the different default.
Useful? React with 👍 / 👎.
|
|
||
| ## Running it | ||
|
|
||
| `./mvnw test -pl :parallel-consumer-example-core -am` (from the repo root). `CoreAppTest` subclasses |
There was a problem hiding this comment.
On a clean checkout, this command cannot resolve parallel-consumer-core:jar:tests: the example declares that classifier as a test dependency, but Maven's test phase stops before the maven-jar-plugin:test-jar goal attached to package. The root pom.xml explicitly records that this artifact does not exist until core is packaged, so an already-installed snapshot merely masks the failure; use package here and in the Reactor and Vert.x example READMEs to run their unit tests without entering the Docker-dependent integration-test phase.
Useful? React with 👍 / 👎.
🧪🔒 Quarantine Lane Report
🔴 expected while the owner PR is open · 🟡🎲 flapper, pass proves nothing · 🚨 a deterministic quarantined test passing means its fix landed: delete its |
Both earn a note on evidence rather than on a branch existing, and both are one unreferenced commit away from being lost. Shared collections across the poll/control boundary. Master still has PCMetrics.registeredMeters as a plain ArrayList, appended per partition inside a rebalance callback and walked-then-cleared in close(); RemovedPartitionState still shares a mutable static TreeSet across every instance in the JVM. Filed as a stall, not a lost metric: the ConcurrentModificationException propagates into doClose's finally and skips the state = CLOSED transition on the next line, so the consumer never departs and the group waits out its session timeout. fix/concurrent-collection-sweep fixes six such sites with a reproduction and a regression test and has no PR - and it touches the files #57 declares it owns, so the sequencing is an open call. The Class 2 chaos RED. test-chaos-phase2.md still says "widen it if it ever flakes"; it flaked, and a seed replay on an uncontended box (154.5s to 121.3s, zero violations) shows the RED was runner contention between Performance and Chaos Pain Suite, not a product stall. Tagged misdirection because that is precisely what it cost: a signature that reads as the exact Class 2 occurrence the hunt wants. The owning note gets a one-line pointer rather than a second copy of the finding. Not written, and why: #315, #313 and #303 are open PRs whose diffs carry their own context; docs/router-ownership-pass, docs/mirror-2023-admin-sweep, docs/plan-package-rename-dry-run and docs/v6-release-ideas-old have all merged or been superseded on master; and docs/297-review-route-and-timing-evidence is superseded by better material already there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019infZnCpPvX8c6QWFC1uLC
Serves no single issue, so none is cited - this is a documentation gap found while working on
#293, not work that issue asked for.
depends on #293
Description
Every Maven module now has a README. Ten did not; two more were fixed in the parent PR. The rule
is the owner's, stated 2026-08-15, and nothing enforces it - the check that finds violations is
recorded in
docs/refactoring.mdalongside the entry this PR resolves.Stacked on
feats/proxy-requirements(#293) so this diff shows only theREADMEs. It is deliberately separate because it is tangential to the language-proxy work and should be
reviewable, revertable and mergeable on its own.
What was written, and how
Each README was written after reading the module, not assembled from its pom and directory names -
a README that reads as documentation while being inferred is worse than none. Where another document
owns a topic it is cross-referenced and its owner named, rather than restated, because a second copy
drifts.
Findings that only came from reading the code, and that no existing document stated:
parallel-consumer-core- ordering decides two independent things, and reading it as one is thecommon mistake: which shard a record lands in (
ShardKey.ofswitches onProcessingOrder), andseparately whether a shard hands out a second record before the first resolves
(
ProcessingShard.isOrderRestricted, false only forUNORDERED). The consequence is tabulated,because it is counterintuitive: under
UNORDEREDthe shard count still equals the partition countwhile concurrency is bounded by the executor pool instead, so shards are the unit of accounting
rather than of parallelism. A second surprise on the first axis: key-ordered shards key on the
topic name, not the topic-partition.
vertx/reactor/mutiny- the three constraints that matter to a user of any of them arecore's, not theirs:
ExternalEnginerejects transactional commit mode outright, so none offersexactly-once; it pins the worker pool to one dispatch thread; and it defers success to async
completion. Written once in core's README and cross-referenced from the other three, rather than
three copies that will disagree within a year.
release.targetcomment rather than re-explained,as are core's junit parallelism property, example-metrics' micrometer pin and example-streams'
jackson CVE floor. Where a pom carries load-bearing reasoning, the README names the comment and
refuses to copy it.
Two documentation defects found and NOT papered over
Both are recorded rather than silently worked around, and both are follow-ups this PR does not fix:
mainmethod, but the rootREADME.adocmodule-maturity table calls them"runnable examples". No run command was invented; each README states that the app is driven by its
own test, and the streams one notes it additionally takes
bootstrapServersvia constructor.src/test/java/**/integrationTests/, notsrc/test-integration/java. Failsafe selects on the package name (**/integrationTest*/**/*.javain the root pom), which
AGENTS.md's one-line suite summary does not cover. The mechanism is statedin the examples parent README, routing to
docs/testing.mdas the owner.Verification
bin/check-copyright-headers.sh- green, 0 violations across 720 files.bin/check-issue-refs.sh- green.Checklist
docs/refactoring.mdcarries the entry it resolvesdocs/features/-N/A - no user-facing behaviour changes;docs/features/is routed to by name as the owner of per-capability recordsN/A - documentation only, no code changes. The claims were verified by reading the source and grepping every citation, which is recorded above