Skip to content

stride_valid assert faild. non-contiguous stride can't handled #10

Open

Description

@study1994

Here is my log with a bug, how can I solve this problem?

[Exception|indice_conv|subm]feat=torch.Size([137566, 16]),w=torch.Size([7, 7, 7, 16, 16]),pair=torch.Size([2, 343, 137566]),pairnum=tensor([ 5001,  4071,  2915,  2876,  2559,  2547,  2521,  4382,  5716,  4867,
         3664,  3167,  3121,  2835,  3526,  5018,  7021,  6005,  4319,  3926,
         3575,  3916,  4679,  6829,  9400,  7201,  5169,  4423,  4106,  4546,
         5253,  7129,  7777,  5756,  3978,  3274,  3504,  3874,  4283,  5309,
         6038,  4567,  2878,  2974,  3229,  3410,  3441,  4449,  5227,  4487,
         3883,  3429,  3430,  3017,  2970,  2785,  4539,  5350,  4998,  4443,
         3730,  3469,  3128,  4072,  5174,  6710,  6427,  4945,  4201,  3716,
         4024,  4918,  6864,  8956,  6847,  5047,  4072,  3896,  4419,  5554,
         7007,  7141,  5674,  4268,  3469,  3878,  4567,  4992,  5347,  5751,
         4694,  3185,  3406,  3888,  4043,  3898,  4522,  4774,  8162,  7692,
         7195,  6938,  6410,  6041,  5768,  8543,  8755,  8271,  7939,  7052,
         6395,  5980,  8122,  9124, 10307, 10506,  8607,  7326,  6615,  8280,
         9333, 11303, 13434, 10871,  8458,  6993,  7694,  8293,  9598, 10743,
         9695,  7938,  6372,  6528,  6814,  7356,  7580,  7401,  7089,  6270,
         5683,  5811,  6181,  6397,  5909,  5726,  5781, 43530, 43224, 40874,
        39942, 38939, 38032, 36751, 43545, 50091, 50867, 47721, 45472, 43093,
        39536, 44726, 52853, 62406, 66190, 56487, 49107, 44248, 47333, 54066,
        66982,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0,     0,     0,     0,     0,     0,     0,     0,
            0,     0,     0], device='cuda:0', dtype=torch.int32),act=137566,algo=ConvAlgo.Native
no apex
No Tensorflow
SPCONV_DEBUG_SAVE_PATH not found, you can specify SPCONV_DEBUG_SAVE_PATH as debug data save path to save debug data which can be attached in a issue.
Traceback (most recent call last):
  File "SpMiddleResNetFHDLargeKernel_forward.py", line 13, in <module>
    ret, multi_scale_voxel_features,_ = center_net(voxel_features, batch_dict, voxel_coords, batch_size, dense_shape)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl-01/HDD/code/FocalsConv/CenterPoint/det3d/models/backbones/scn_largekernel.py", line 290, in forward
    x_conv1 = self.conv1(x)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/spconv/pytorch/modules.py", line 138, in forward
    input = module(input)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl-01/HDD/code/FocalsConv/CenterPoint/det3d/models/backbones/scn_largekernel.py", line 181, in forward
    out = self.conv1(x)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl-01/HDD/code/FocalsConv/CenterPoint/det3d/models/backbones/scn_largekernel.py", line 92, in forward
    x_conv_block = self.block(x_conv)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/spconv/pytorch/conv.py", line 755, in forward
    return self._conv_forward(self.training,
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/spconv/pytorch/conv.py", line 327, in _conv_forward
    out_features = Fsp.indice_subm_conv(
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/torch/cuda/amp/autocast_mode.py", line 118, in decorate_fwd
    return fwd(*args, **kwargs)
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/spconv/pytorch/functional.py", line 327, in forward
    raise e
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/spconv/pytorch/functional.py", line 308, in forward
    return ops.indice_conv(features,
  File "/home/dl-01/HDD/anaconda3/envs/openpcdet/lib/python3.8/site-packages/spconv/pytorch/ops.py", line 860, in indice_conv
    ConvGemmOps.indice_conv(alloc, ext_mm, GEMM_CPP, ALL_WEIGHT_IS_KRSC,
RuntimeError: /tmp/pip-build-env-h7b1kdx5/overlay/lib/python3.8/site-packages/cumm/include/tensorview/tensor.h(770)
stride_valid assert faild. non-contiguous stride can't handled.

Activity

  1. lda1049187465 commented on Jun 13, 2023

    @lda1049187465

    I have the same problem.Did you solve it?

  2. yukang2017 commented on Jul 14, 2023

    @yukang2017
    Member

    Sorry for the late reply. Would you please provide the version information, torch and spconv?

  3. Karkers commented on Dec 10, 2023

    @Karkers

    相同的问题,torch1.10,cu113

  4. Tream733 commented on Apr 17, 2024

    @Tream733

    I have the same problem. torch 1.12.0+cu116 ; spconv-cu116 2.3.6

  5. Tream733 commented on Apr 17, 2024

    @Tream733

    Sorry for the late reply. Would you please provide the version information, torch and spconv?
    it's work. #11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions