gh-150723: Fix perf jitdump files on macOS#150728
Merged
Merged
Conversation
The perf jitdump format defines the thread id field of the JR_CODE_LOAD record as a 32-bit value, but on macOS it was declared as a uint64_t (since pthread_threadid_np() returns a uint64_t). Those extra 8 bytes plus alignment padding shifted every following field, so parsers reading the file by the spec misread code_size as the code address and failed to resolve any Python frames. Declare thread_id as uint32_t on all platforms and truncate the macOS thread id when writing the record. The value is only informational. Symbols are resolved by address, and not thread ids so truncation is safe here.
Contributor
Author
|
Hi @pablogsal! Would you mind reviewing this PR? |
On macOS the jitdump file is consumed by profilers such as samply, which timestamp their samples using mach_absolute_time(). The jitdump events were stamped with clock_gettime(CLOCK_MONOTONIC), a different clock domain that keeps advancing while the system is asleep, so the JIT code mappings could be off by days relative to the samples and no Python frame would resolve. Stamp jitdump events with mach_absolute_time() on macOS so they share the sampler's clock domain. Linux continues to use CLOCK_MONOTONIC to stay aligned with perf.
Exercise the -Xperf_jit (jitdump) backend through samply and assert that Python frames resolve, exercising the binary jitdump path end to end. Skipped when samply is not installed.
pablogsal
approved these changes
Jun 2, 2026
Member
pablogsal
left a comment
pablogsal
left a comment
There was a problem hiding this comment.
I built and tested this on macOS arm64 with samply 0.13.1.
Checks run:
./configure --with-pydebugmake -j8./python.exe -m test test_samply_profiler -v- manual
samply record --save-onlyagainst./python.exe -Xperf_jit, verifyingpy::foo:<string>,py::bar:<string>, andpy::baz:<string>resolved in the captured profile
The jitdump layout/timestamp changes look correct to me, and the new regression coverage exercises the failing path end to end. Approved.
Member
|
Thanks a lot for looking into this @canova |
pablogsal
added
the
needs backport to 3.15
|
Thanks @canova for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15. |
|
GH-150832 is a backport of this pull request to the 3.15 branch. |
bedevere-app
Bot
removed
the
needs backport to 3.15
pablogsal
pushed a commit
that referenced
this pull request
Jun 3, 2026
gh-150723: Fix perf jitdump files on macOS (GH-150728) The perf jitdump format defines the thread id field of the JR_CODE_LOAD record as a 32-bit value, but on macOS it was declared as a uint64_t (since pthread_threadid_np() returns a uint64_t). Those extra 8 bytes plus alignment padding shifted every following field, so parsers reading the file by the spec misread code_size as the code address and failed to resolve any Python frames. Declare thread_id as uint32_t on all platforms and truncate the macOS thread id when writing the record. The value is only informational. Symbols are resolved by address, and not thread ids so truncation is safe here. * Use mach_absolute_time for macOS jitdump timestamps On macOS the jitdump file is consumed by profilers such as samply, which timestamp their samples using mach_absolute_time(). The jitdump events were stamped with clock_gettime(CLOCK_MONOTONIC), a different clock domain that keeps advancing while the system is asleep, so the JIT code mappings could be off by days relative to the samples and no Python frame would resolve. Stamp jitdump events with mach_absolute_time() on macOS so they share the sampler's clock domain. Linux continues to use CLOCK_MONOTONIC to stay aligned with perf. Exercise the -Xperf_jit (jitdump) backend through samply and assert that Python frames resolve, exercising the binary jitdump path end to end. Skipped when samply is not installed. (cherry picked from commit 494f2e3) Co-authored-by: Nazım Can Altınova <canaltinova@gmail.com>
Contributor
Author
|
Thanks a lot for the quick review @pablogsal! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Python's jitdump output was unusable on macOS, thanks to @not-matthias who noticed it. Profilers like samply showed every Python frame as a raw hex address. There were two separate bugs, both in how python writes this jitdump file:
The
thread_idfield inJR_CODE_LOADrecords was written as a 64-bit value instead of the 32-bit value the format requires. The extra bytes shifted every field after it, so parsers misread the records.jitdump timestamps used
clock_gettime(CLOCK_MONOTONIC). That works on Linux becauseperftimestamps its samples with the same clock, but macOS profilers usemach_absolute_time(), which is a different clock domain. The two can drift apart by days, so the JIT mappings never lined up with the samples and nothing resolved before. On macOS we now usemach_absolute_time(). Linux keepsCLOCK_MONOTONIC.With both fixes, samply resolves Python frames from the jitdump end to end.
Note that
PYTHONPERFSUPPORT=1was working since it doesn't use jitdumps.PYTHON_PERF_JIT_SUPPORT=1was broken.I also added a test that runs the
-Xperf_jitbackend through samply and checks that the frames show up (it skips when samply isn't installed similary to the other tests I added before). The existing samply test only covered the-Xperfperf-map path, which wasn't broken.