Repository navigation
Fused types in Cython cause line_profiler to not measure any timing #213
Description
I tested using the same code as the collatz example and line_profiler<4.
@cython.binding(True)
def collatz1(np.uint64_t n):
while n > 1:
if n % 2 == 0:
n //= 2
else:
n = 3*n+1
ctypedef fused fused_t:
np.int32_t
np.int64_t
@cython.binding(True)
def collatz2(fused_t n):
while n > 1:
if n % 2 == 0:
n //= 2
else:
n = 3*n+1func = collatz1
profile = line_profiler.LineProfiler(func)
profile.runcall(func, 19)
assert_stats(profile, func.__name__)
func = collatz2
profile = line_profiler.LineProfiler(func)
profile.runcall(func, 19)
assert_stats(profile, func.__name__)I get the following output:
Total time: 1.6e-05 s
File: collatz.pyx
Function: collatz1 at line 11
Line # Hits Time Per Hit % Time Line Contents
==============================================================
11 def collatz1(np.uint64_t n):
12 1 6.0 6.0 37.5 while n > 1:
13 20 5.0 0.2 31.2 if n % 2 == 0:
14 14 3.0 0.2 18.8 n //= 2
15 else:
16 6 2.0 0.3 12.5 n = 3*n+1
Timer unit: 1e-06 s
Total time: 0 s
File: collatz.pyx
Function: collatz2 at line 23
Line # Hits Time Per Hit % Time Line Contents
==============================================================
23 def collatz2(fused_t n):
24 while n > 1:
25 if n % 2 == 0:
26 n //= 2
27 else:
28 n = 3*n+1
Traceback (most recent call last):
File "test_profile.py", line 33, in <module>
assert_stats(profile, func.__name__)
File "test_profile.py", line 16, in assert_stats
assert len(timings) > 0
AssertionError
Activity
Erotemic commentedon May 12, 2023
Is the claim that this worked before line profiler 4.0? Or is the issue that this never worked? I don't think we directly support cythonized code at the line level. @Theelx, do you have any insight here?
louisabraham commentedon May 13, 2023
You do support Cython code at the line level: https://github.com/cython/cython/blob/master/tests/run/line_trace.pyx
It requires
line_profiler<4.Maybe the bug is in Cython, do you want me to post an issue there?
[-]Fused types cause line_profiler to not measure any timing[/-][+]Fused types in Cython cause line_profiler to not measure any timing[/+]Erotemic commentedon May 13, 2023
No, its likely the bug is here. The issue with the 3.x branch is it doesn't support 3.11+. I'm not sure what is needed to re-add this capability. I'm not the original author of the code, I'm just the guy who maintains it. Fixing this issue will requires someone to dig into this and understand what's going on.
Theelx commentedon May 14, 2023
Hey, sorry for the late response, I've been super busy with my midterms and finals schedule lately (finishing up my first year of college!). I'll take a look at this and the other issues in a few days, once my last final has been taken.
Theelx commentedon May 17, 2023
So, fused types in cython generate multiple specialized versions of the code, one for each possible type. I'd be surprised if profiling had ever worked for fused types, since one set of source code maps to multiple different compiled versions. If profiling fused types had worked on line_profiler<4, then I would put effort into fixing it, but since that wasn't the case I think it would take a lot of development time for a fairly niche case.