[Fix] Fix get_valid_count flaky test for cuda - #4901
Conversation
trevor-m
left a comment
trevor-m
left a comment
There was a problem hiding this comment.
Thanks for fixing this!
|
ping @kevinthesun @yzhliu |
* get_valid_count accuracy issue fixed for individual tests but not for all tests running together * minor fix * initialize valid_count and PrefixSum buffers * test updated * udpate relay test as well * update document * fix lint * address comment * fix lint * correct atomicAdd identifier name
* get_valid_count accuracy issue fixed for individual tests but not for all tests running together * minor fix * initialize valid_count and PrefixSum buffers * test updated * udpate relay test as well * update document * fix lint * address comment * fix lint * correct atomicAdd identifier name
* get_valid_count accuracy issue fixed for individual tests but not for all tests running together * minor fix * initialize valid_count and PrefixSum buffers * test updated * udpate relay test as well * update document * fix lint * address comment * fix lint * correct atomicAdd identifier name
|
@Laurawly I still get a flaky failure from get_valid_count test in my PR |
|
@masahi That’s weird. You can comment off the test for now and I’ll try to reproduce it on my end. |
|
@Laurawly Also reported at a different PR #4931 (comment) |
Yeah, pls feel free to comment out the test: https://github.com/apache/incubator-tvm/blob/master/topi/tests/python/test_topi_vision.py#L106 |
|
fortunately my PR is green now without commenting out :) If the next open PR by somebody else sees the same problem, I'll ask him/her to comment it out. |
5 participants
Turned on get_valid_count test for cuda in topi. Used atomic operations in this fix to replace previous block sync method.
Tested on V100 and T4 GPUs.
@trevor-m @kevinthesun @yzhliu Could you review?