Repository navigation
fixes #29: Faster record producing - removes blocks on Future and processes results asynchronously - #356
Closed
Antony Stubbs (astubbs) wants to merge 14 commits into
Conversation
Antony Stubbs (astubbs)
changed the title
| * records only affect things at commit time - that's great. | ||
| */ | ||
| protected void handleFutureProduceResultsAsync(List<FutureConsumeProduceResult<K, V, K, V>> results) { | ||
| getMyActor().tell(controller -> handleFutureProduceResultsMessage(results)); |
Contributor
Author
There was a problem hiding this comment.
Can the result straight into a concurrent queue, instead of blocking on actor work. However it’s only a performance upgrade, so actor work may be higher priority
Antony Stubbs (astubbs)
changed the base branch from
master
to
improvements/transaction-docs
July 16, 2022 06:58
…stead of blocking get on future
Antony Stubbs (astubbs)
force-pushed
the
improvements/async-process-send-results
branch
from
July 16, 2022 07:19
00f3501 to
9b91d57
Compare
… list of expected records to produce, and count them down as they finish, when they do, onSuccess each wc. Issue is that our api doesn't connect in records without records in the batching API, so we don't know which out record corresponds with which WC. We only know when our expected sent count reaches actual sent count. There is still some questions to answer here with how this interacts with transactions, but as long as the transaction doesn't commit until everything's complete. And new records are blocked, and retrying any failures doesn't result in dupes which it shouldn't.
Antony Stubbs (astubbs)
force-pushed
the
improvements/transaction-docs
branch
from
August 31, 2022 13:11
978ed08 to
b2be8da
Compare
Contributor
|
Closing - Stale. |
4 tasks done
Antony Stubbs (astubbs)
referenced
this pull request
in astubbs/parallel-consumer
Aug 6, 2026
…e's blind spot Review found the fourth instance of the pattern a few lines below the third fix. Swept the whole file this time instead of catching one more instance, using the gate's own stripQualified() so "unqualified" means exactly what CI means by it. Twenty-four references in the closed-upstream-PR catalogue were bare and are now written out and hyperlinked. The trap that makes this worth the verbosity: upstream #356's own title is "fixes #29: Faster record producing", and a bare #29 here autolinks to FORK #29, which is the paused-consumption-after-rebalance fix. Same number, unrelated work. Three more the review did not spot, found by sweeping: - L132, L144: "#200" means upstream #200 (shared-nothing), but fork #200 exists - it is docs(build) #180 about ManagedTruth. The gate passes this, because the number resolves. It just resolves to the wrong issue. - L224: "#233" means upstream #233; no fork #233 exists, so the gate would have caught this one had it been an added line. That asymmetry is now documented in AGENTS.md: the gate flags bare numbers that FAIL to resolve, so a wrong reference that happens to resolve sails through, and looks fine. As fork numbering grows the collisions increase, so this gets worse rather than better. Verified the remaining bare numbers in the file (#143, #131, #101, #73, #57, #110, #117, #142, #40) are all genuine fork references. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QqHpNSXC39ANv9kG1ZvUzn
4 tasks done
Antony Stubbs (astubbs)
added a commit
to astubbs/parallel-consumer
that referenced
this pull request
Aug 18, 2026
…ied directions (#310) Adds the ideation deliverable for reviving the 2022 micro-actor branch family (manifest entry sweep-2023-actor-ipc, registered by #305), plus docs/inflight/next-actor-revival.md so the study is discoverable from the inflight directory and names the open decision. The doc (docs/ideation/2026-08-17-actor-collection-revival-ideation.html, self-contained HTML) holds six survivors from a wide generated field, each with a verified basis - direct quotes spot-checked against the repo and the 2022 branch trees by an independent verification pass - plus confidence, complexity, downsides, and a rejection table recording what was cut and why, including one candidate killed because its cited evidence was factually backwards and one whose headline number was a diffstat misread. Ranked survivors: 1. One-day falsification pipeline - rename-script the four framework files, compile standalone, conformance-test the documented FILO invariant; converts the two standing editorial verdicts on the family ("only meaningful as part of confluentinc#200", "far too stale to apply directly") from opinion into measurement 2. Un-bundle the family - async-produce (confluentinc#356) is ranked throughput-critical, not architecture, and its target defect is verified still in master source 3. Controlled experiment on the confluentinc#857 commit path - mailbox arm vs #29's lock arm, judged by the recorded chaos replay seeds 4. Accession and graded register - the manifest half landed via #305; per-branch readiness grades remain open 5. Skeleton-first strangler - land the six thread-ownership interfaces as a pure refactor; the mailbox becomes a per-seam swap 6. Concurrency mass budget - an ArchUnit ratchet on primitive counts The inflight note records the open decision: the doc's top pick is 1; the real fork in the road is 3 vs 5. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Performance improvement for sending records.
Checklist
Blocked by: