Skip to content

fix: concurrent access to discovery sync data listeners(#6686) - #7197

Open
juicewcode wants to merge 8 commits into
apache:masterfrom
juicewcode:fix/6686-discovery-list-thread-safety
Open

juicewcode wants to merge 8 commits into
apache:masterfrom
juicewcode:fix/6686-discovery-list-thread-safety

Conversation

@juicewcode

Copy link
Copy Markdown
Contributor

Fixes #6686

Summary

  • Replace ArrayList with CopyOnWriteArrayList in DiscoveryDataChangedEventSyncListener.
  • Add a deterministic concurrency test reproducing the issue scenario.

Test

  • Added testOnChangeIsSafeWhenListenerIsAddedConcurrently.
  • The test pauses onChange while processing a listener, adds another listener concurrently, then verifies that
    onChange completes without throwing an exception.

  Replace the non-thread-safe ArrayList in DiscoveryDataChangedEventSyncListener with CopyOnWriteArrayList and add a
  deterministic concurrency test covering listener registration during onChange iteration.

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right fix for a real concurrency hazard.

onChange iterates discoverySyncDataList while addListener can append to it on another thread (a discovery handler being registered mid-sync). With a plain ArrayList that produces ConcurrentModificationException — or, worse, silently skips a listener. CopyOnWriteArrayList is the correct choice here because the read path (iteration on every change event) dominates and writes happen rarely, so the copy cost is irrelevant.

Verified:

  • discoverySyncDataList is only mutated through addListener, so the snapshot semantics of COW give exactly what onChange needs: a stable view for the duration of one event.
  • The new test reproduces the race properly rather than just asserting the field type — it parks keyValueParser.parseValue mid-flight, calls addListener from the main thread while the worker is inside onChange, and then asserts the worker completes without throwing. That's a genuine regression test for CME.
  • Latch cleanup in the finally block avoids hanging the suite if the assertion fails.

Non-blocking note: COW iteration is a snapshot, so a listener added during an onChange won't be picked up by that particular event. That's the correct trade-off (better to skip one event than to throw), but it's worth a brief comment on the field so the semantics are explicit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] DiscoveryDataChangedEventSyncListener.discoverySyncDataList is a non-thread-safe ArrayList mutated across threads

2 participants