Skip to content

gh-154756: Fix data race in list.sort() - #154572

Open
weixlu wants to merge 1 commit into
python:mainfrom
weixlu:list
Open

gh-154756: Fix data race in list.sort()#154572
weixlu wants to merge 1 commit into
python:mainfrom
weixlu:list

Conversation

@weixlu

@weixlu weixlu commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

This PR try to fix #154756

The fix is straightforward: publish each slot write with FT_ATOMIC_STORE_PTR_RELEASE, just like the other list mutators already do. For example, ins1() (used by list.insert) also uses FT_ATOMIC_STORE_PTR_RELEASE(items[i+1], items[i])

Testing

  • The reproducer now runs clean.
  • PASS ./python -m test test_sort test_list

Notes for reviewer:

  • On the default build (GIL-enabled), FT_ATOMIC_STORE_PTR_RELEASE expands to a plain store, so there is no change there.
  • This PR only adds a atomic store, so basic functional behavior is unchanged.

@brijkapadia

Copy link
Copy Markdown
Contributor
  1. Could you create a new issue instead of referencing the parent?
  2. Are you sure using a lot of release atomics is the best approach?

@weixlu weixlu changed the title gh-153852: Fix data race in list.sort() gh-154756: Fix data race in list.sort() Jul 27, 2026
@weixlu

weixlu commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author
  1. Could you create a new issue instead of referencing the parent?
  2. Are you sure using a lot of release atomics is the best approach?

@brijkapadia Thanks for your review. Sure, I’ve created a separate issue.

You’re right that this PR introduces quite a few release atomic stores. However, I think most of them are necessary to preserve the required atomicity.

  • listobject.c already contains 19 release atomic stores used by methods such as list.clear() and list.extend(), and I believe list.sort() requires similar changes.
  • That said, 6 atomic stores can be removed. They are in the original ordinary insertion sort implementation, which has been superseded by binary insertion sort and is currently guarded by #if 0. Since this code no longer appears to be used, there is probably no need to update it. I initially added the atomic stores there as well simply to keep the implementations consistent.

@picnixz

picnixz commented Jul 27, 2026

Copy link
Copy Markdown
Member

What is the performance impact for lists of various size?

@weixlu

weixlu commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

What is the performance impact for lists of various size?

@picnixz Hi, I have run sortperf.py on a --disable-gil --enable-optimizations build

  • For random input, it's 5%-7% slower
  • For already-ordered / all-equal / worst-case inputs, it has little impact.

Random data (list_sort) across sizes

size before after ratio
16 180 ns 198 ns 1.10x slower
64 944 ns 1.08 us 1.14x slower
256 5.86 us 6.21 us 1.06x slower
1024 52.6 us 55.2 us 1.05x slower
4096 330 us 354 us 1.07x slower
16384(default) 1.70 ms 1.83 ms 1.07x slower
65536 8.81 ms 9.42 ms 1.07x slower
262144 51.8 ms 54.3 ms 1.05x slower

All input patterns at size = 16384 (default)

benchmark before after ratio
list_sort (random) 1.70 ms 1.83 ms 1.08x slower
list_sort_ascending not significant
list_sort_ascending_exchanged not significant
list_sort_ascending_one_percent not significant
list_sort_ascending_random 127 us 123 us 1.03x faster
list_sort_descending 158 us 143 us 1.10x faster
list_sort_duplicates 323 us 343 us 1.06x slower
list_sort_equal not significant
list_sort_worst_case 122 us 124 us 1.02x slower

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Data race in list.sort() on no-gil build

3 participants