Repository navigation
BUG: LineProfiler.__call__() bugged on class and static methods #328
Description
Summary
In the discussion around #326, it came up that the below (existing) implementation of line_profiler[.line_profiler].LineProfiler.wrap_classmethod() (which is used by LineProfiler.__call__() (i.e. @profile) when decorating class methods) is bugged:
def wrap_classmethod(self, func):
"""
Wrap a classmethod to profile it.
"""
@functools.wraps(func)
def wrapper(*args, **kwds):
self.enable_by_count()
try:
result = func.__func__(func.__class__, *args, **kwds)
finally:
self.disable_by_count()
return result
return wrapperand can be problematic in certain edge cases because it replaces a classmethod with a types.FunctionType. The same also goes for staticmethod, which currently doesn't have a dedicated wrap_*() method, and is thus dispatched to wrap_{function,coroutine,generator}() with similar caveats.
What does work now
The bug has so far evaded detection, because:
- It does not affect (whole-file) autoprofiling, which rewrites the module AST to put
@profileat the bottom of the decorator stack of function definitions. That is to say,@profilewould come before any@classmethodor@staticmethod, and thuswrap_classmethod()is circumvented. The same of course goes for when the user use explicit decoration and place@profilein the same place. - It does not affect the use-case of just plucking a class method from a class, like:
because accessing a class method from a class returns a bound method (
class Class: @classmethod def method(cls, *args, **kwargs): ... lp = LineProfiler() wrapped_call_result = lp(Class.method)(*args, **kwargs)
types.MethodType), which is then presumably dispatched towrap_function(). Notably, this is the use-case covered bytests/test_line_profiler.py::test_classmethod_decorator().
Either way, LineProfiler.wrap_classmethod() is seldom actually used, and is untouched by the test suite.
What doesn't work
However, should the user mess up the decorator order and appliy @profile to a classmethod, the returned object binds wrongly on class instances since it's just a normal function:
Failing example (click to expand)
(py3.13) $ cat test_classmethod.py
from line_profiler import LineProfiler
class MyClass:
@profile
@classmethod
def my_method(cls) -> str:
return '{0.__module__}.{0.__qualname__}'.format(cls)
@profile
def main():
obj = MyClass()
MyClass.my_method() # This is apparently OK, but...
obj.my_method() # ... this errors out
if __name__ == '__main__':
main()
(py3.13) $ kernprof -lv test_classmethod.py
<MYENV>/line_profiler/line_profiler/line_profiler.py:75: UserWarning: Adding a function with a __wrapped__ attribute. You may want to profile
the wrapped function by adding my_method.__wrapped__ instead.
self.add_function(func)
Wrote profile results to test_classmethod.py.lprof
Timer unit: 1e-06 s
Total time: 3e-06 s
File: test_classmethod.py
Function: my_method at line 5
Line # Hits Time Per Hit % Time Line Contents
==============================================================
5 @profile
6 @classmethod
7 def my_method(cls) -> str:
8 1 3.0 3.0 100.0 return '{0.__module__}.{0.__qualname__}'.format(cls)
Total time: 1.2e-05 s
File: test_classmethod.py
Function: main at line 11
Line # Hits Time Per Hit % Time Line Contents
==============================================================
11 @profile
12 def main():
13 1 1.0 1.0 8.3 obj = MyClass()
14 1 7.0 7.0 58.3 MyClass.my_method() # This is apparently OK, but...
15 1 4.0 4.0 33.3 obj.my_method() # ... this errors out
Traceback (most recent call last):
File "<MYENV>/py3.13/bin/kernprof", line 8, in <module>
sys.exit(main())
~~~~^^
File "<MYENV>/line_profiler/kernprof.py", line 381, in main
execfile(script_file, ns, ns)
~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "<MYENV>/line_profiler/kernprof.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), globals, locals)
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "test_classmethod.py", line 18, in <module>
main()
~~~~^^
File "<MYENV>/line_profiler/line_profiler/line_profiler.py", line 150, in wrapper
result = func(*args, **kwds)
File "test_classmethod.py", line 15, in main
obj.my_method() # ... this errors out
~~~~~~~~~~~~~^^
File "<MYENV>/line_profiler/line_profiler/line_profiler.py", line 94, in wrapper
result = func.__func__(func.__class__, *args, **kwds)
TypeError: MyClass.my_method() takes 1 positional argument but 2 were given
(py3.13) $ What's more, it's also incorrect even when called from the class, because the line result = func.__func__(func.__class__, *args, **kwds) binds classmethod instead of the owning class to the first argument.
Since staticmethod doesn't have a dedicated wrap_staticmethod() method, when decorating them in the above manner they are likewise reduced to a function, and thus suffer from similar argument-binding issues.
Recommendation
Maybe something like this:
def wrap_classmethod(self, func):
"""
Wrap a classmethod to profile it.
"""
return classmethod(self(func.__func__))
def wrap_staticmethod(self, func):
"""
Wrap a staticmethod to profile it.
"""
return staticmethod(self(func.__func__))Will probably write a PR, but there are already several in the pipeline currently.
Activity