Repository navigation
Improvements/transaction docs large refactorings - #399
Merged
Antony Stubbs (astubbs) merged 15 commits intoSep 2, 2022
Conversation
…ocs-large-refactorings
6 tasks done
Antony Stubbs (astubbs)
changed the base branch from
master
to
improvements/di-module
August 25, 2022 10:36
…ocs-large-refactorings # Conflicts: # CHANGELOG.adoc # README.adoc # parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/internal/PCModule.java # parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/internal/PCModuleProd.java
Antony Stubbs (astubbs)
left a comment
Antony Stubbs (astubbs)
left a comment
Contributor
Author
There was a problem hiding this comment.
LGTM
Antony Stubbs (astubbs)
marked this pull request as ready for review
August 25, 2022 11:14
DI-Module, very useful for testing
…ction-docs-large-refactorings # Conflicts: # CHANGELOG.adoc # README.adoc # parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/ParallelEoSStreamProcessor.java # parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/internal/PCModule.java # parallel-consumer-core/src/test-integration/java/io/confluent/parallelconsumer/integrationTests/TransactionMarkersTest.java
Roman Kolesnev (rkolesnev)
approved these changes
Sep 2, 2022
Roman Kolesnev (rkolesnev)
left a comment
Roman Kolesnev (rkolesnev)
left a comment
Contributor
There was a problem hiding this comment.
LGTM - couple of naming / flow comments above.
Antony Stubbs (astubbs)
deleted the
improvements/transaction-docs-large-refactorings
branch
September 2, 2022 12:18
Antony Stubbs (astubbs)
restored the
improvements/transaction-docs-large-refactorings
branch
September 2, 2022 12:23
Antony Stubbs (astubbs)
added a commit
that referenced
this pull request
Sep 2, 2022
Antony Stubbs (astubbs)
added a commit
that referenced
this pull request
Sep 2, 2022
) - Cohesion FTW (Merged from PR: Fix for merge error (#406))
Antony Stubbs (astubbs)
deleted the
improvements/transaction-docs-large-refactorings
branch
September 2, 2022 13:13
Antony Stubbs (astubbs)
added a commit
that referenced
this pull request
Sep 29, 2022
* Transactional commit mode system improvements and docs (#355) ** Clarifies transaction system with much better documentation. ** Fixes a potential race condition which could cause offset leaks between transactions boundaries. ** Introduces lock acquisition timeouts. ** Fixes a potential issue with removing records from the retry queue incorrectly, by having an inconsistency between compareTo and equals in the retry TreeMap. Details: * Fix potential race condition with tx committing and work being available by expanding locking scope Don't finish producing with lock until work as been put into work inbox so that controller can process it before arranging commits. Aquire the commit lock earlier, so that no more records are able to be sent between process results and committing transaction. Otherwise a record from a future commit coule be added to this transaction, without its source offsets being collected for commit due to the work's status not being completed (as it hasn't been processed by the controller yet). * Too many messages were being produced at a certain times, making the test unreliable. Changed system to add only minimum amount needed, by calculating via bytes required to add to payload. * eliminate race condition by waiting for commit lock to be fully released before trying to acquire * refactor: Refactoring split ProducerManager out to ProducerWrapper (#399) * Remove sync lock on producer wrapper for committing transactions - shouldn't be needed anymore with the more robust produce lock - as that blocks producer access while committing instead of relying on sync locks * Create single point for integration commit assertions * Fixed an issue where a WC may not get removed from the retry queue due to inconsistency between equals and compareTo in TreeMap * Turn off forced transaction commit mode for vertx test - can't see why it had been done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Blocked by: