Skip to content

Dead letter queue (DLQ) as option for retry expiration - #366

Closed
Antony Stubbs (astubbs) wants to merge 30 commits into
confluentinc:masterfrom
astubbs:features/retry-dlq
Closed

Antony Stubbs (astubbs) wants to merge 30 commits into
confluentinc:masterfrom
astubbs:features/retry-dlq

Conversation

@astubbs

@astubbs Antony Stubbs (astubbs) commented Jul 27, 2022 •

Copy link
Copy Markdown
Contributor

Description...

Checklist

  • Documentation (if applicable)
  • Changelog

Blocked by:

Antony Stubbs (astubbs) and others added 30 commits May 6, 2022 13:11
…ption

# Conflicts:
#	parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/internal/AbstractParallelEoSStreamProcessor.java
#	parallel-consumer-core/src/test/resources/logback-test.xml
@astubbs
Antony Stubbs (astubbs) changed the base branch from features/retry-exception to master July 27, 2022 11:56
@astubbs Antony Stubbs (astubbs) linked an issue Jul 27, 2022 that may be closed by this pull request
@eddyv

Copy link
Copy Markdown
Contributor

Closing - Stale.

Antony Stubbs (astubbs) added a commit to astubbs/parallel-consumer that referenced this pull request Aug 14, 2026
…duce-send failure

Raised while reading back the #261 fix. The surviving throw in
ProducerManager's send callback is reachable in ordinary operation - client-side
rejection of a result record the user's function returned, RecordTooLargeException
being the case the new IT exercises - so it is an expected state, not an internal
fault. Calling it InternalRuntimeException makes every reader re-derive that whole
chain (which path is reachable, why the async one is not, what actually throws)
before concluding it is ordinary failure handling. That derivation was done from
source and kafka-clients bytecode during review and was recorded nowhere.

Preferred fix is non-breaking: throw a specific subclass so existing
catch (InternalRuntimeException) keeps working while the type states the
situation. Renaming the base type would be user-visible and is called out as
belonging in the breaking-changes section instead.

Also corrects this note's own account of confluentinc#291. It is a PR, not an
issue, it implements confluentinc#242, and both concern exceptions the USER
FUNCTION throws - not send-failure classification. Reading them as cover for
send retries was wrong; that work sits under #141 / #149, epic
#239.

Records that both are already accounted for, so neither gets re-mirrored:
confluentinc#291 was closed unmerged on 2023-06-15, in the PR half of the
administrative sweep, and is carried in upstream-map.yaml under
sweep-2023-admin-closure - it has no dedicated fork issue only because that
cohort mirrored the 28 swept issues, with the 35 swept PRs held in the map.
confluentinc#242 is not sweep-affected at all, closed as completed by astubbs in
2022. Notes the one real gap: #239's "Prior work" cites confluentinc#366
from that same cohort but not confluentinc#291, which is equally prior art for
the retry half of that epic.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a dead letter queue (DQL) implementation

3 participants