Repository navigation
missing stuffs in torch.nn.utils #1243
Description
- clip_grad_norm_
- clip_grad_value_
- convert_conv2d_weight_memory_format
- fuse_conv_bn_eval
- fuse_conv_bn_weights
- fuse_linear_bn_eval
- fuse_linear_bn_weights
- parameters_to_vector
- parametrizations
- remove_spectral_norm
- remove_weight_norm
- rnn
- skip_init
- spectral_norm
- stateless
- vector_to_parameters
rnnis checked because the class do exist in TorchSharp, but actually not all the methods inside have been implemented.
Reacted by GeorgeS2019
Activity
shaltielshmid commentedon Feb 27, 2024
Hi @yueyinqiu !
Thanks for writing up the diff. I went through all the methods/modules that were missing in TorchSharp, and they are all pure PyTorch code that don't appear in LibTorch (the underlying C++ library). This means that we would need to rewrite all the methods ourselves in TorchSharp, which can take time to write properly.
Do you have any specific methods that are more important for you in the short term? If so, I can dedicate some time to porting those over in the shorter term.
All contributions are more than welcome, so if you want to port some of the functions as well, that would be great!
[Side note: The
clip_grad_normfunction in PyTorch is deprecated and just calls theclip_grad_norm_function, which exists in TorchSharp as well].yueyinqiu commentedon Feb 28, 2024
Well in fact I'm just a newbie to deep learning and I was trying reproducing some others' work which used
spectral_norm. But I have later found that they skipped all thespectral_norms in their final configuration. So it's not that urgent and I'm just making a note here.Actually I'd like to contribute the project. It's so great to be able to use C# instead of Python. However I'm afraid that my unfamiliarity with deep learning and PyTorch will mess everything up :(
(Please feel free to edit my list, like just remove the
clip_grad_normhere if you think it's appropriate.)yueyinqiu commentedon Mar 7, 2024
I've tried to add
fuse_conv_bn_weightsandfuse_linear_bn_weightsin #1262 . Could you please take a look and give me some advice?NiklasGustafsson commentedon Mar 15, 2024
@yueyinqiu -- I just merged your PR. Please edit the to-do list in the first comment to reflect your changes.
yueyinqiu commentedon Mar 16, 2024
yes and thanks a lot
NiklasGustafsson commentedon Jun 19, 2024
@yueyinqiu -- I was taking a look at this again -- the fuse_***_eval methods, how commonly used are those? They seem straight-forward enough, but how important are they to have?
yueyinqiu commentedon Jun 20, 2024
Hmm... I'm not sure. I'm not really familiar with deep learning. But implementing this won't cost a lot, I think?
Are you considering this because of #1259, and hesitating whether to have generalized types for
Convs andBatchNorms? I think those types would be great, but it's really hard to design them well, and it won't bother too much to use overloads if we don't have such general types.NiklasGustafsson commentedon Jun 20, 2024
fuse_linear_bn_evalandfuse_conv_bn_evalwill be easier to implement once we have the restructured module implementations merged, so I'll hold off until then.yueyinqiu commentedon Jun 20, 2024
I think that would be a good idea.
alinpahontu2912 commentedon Feb 16, 2026
Hi @yueyinqiu!
Thank you for bringing up the missing functionalities in
torch.nn.utils, and for your contributions to the discussion over the past couple of years. It looks like some methods are still pending, particularlyconvert_conv2d_weight_memory_format,fuse_conv_bn_eval, and others.Since TorchSharp has evolved significantly since this issue was opened, I wanted to check in and see if this issue is still relevant or if it has been resolved in the meantime. If you have any updated information or insights, please let us know!
Thanks again for your involvement, and we appreciate your patience!
yueyinqiu commentedon Feb 17, 2026
As far as I know, these functions haven't been implemented yet. I recall that we were in the middle of a major refactor #1259 at the time, which would expose more properties at the C# layer. Since those properties are essential for implementing these functions, we decided to wait for that refactor to be completed.
#1259 is actually a foundational change that impacts many other areas of the project, though I can't recall for certain if it was ever fully merged or finished. It might be worth checking the status of #1259 first. (That specific PR was closed—possibly to avoid taxing the CI/CD at the time.) The spreadsheet in #1271 (comment) may also help.