Apple Silicon support

When planning your roadmap, please consider adding apple silicon support. I know that adding support is not trivial, but I think there’s value in supporting the platform.

Apple silicon support is important to allow TorchSharp use in an educational setting. My unscientific survey is that ~50% of students in university classes use apple silicon. It would also make it easier for downstream libraries such as Diffsharp¹ and ML.NET's new TextClassification trainer to support Apple silicon.

libtorch-cpu on apple silicon would be sufficient, but gpu-acceleration is also available now in PyTorch 1.12.

It is possible to use the DiffSharp reference backend on apple silicon. ↩

Maybe same with #643

We haven't upgraded to 1.12 yet -- does the Pytorch backend support Mac silicon? It didn't in 1.11, AFAIK. TorchSharp is just a wrapper on top of libtorch.dylib on MacOS, so any silicon support will come from the native implementation.

import torch

mps = torch.device("mps")
print(torch.cuda.is_available())
print(mps)

x = torch.ones(1).to(mps)
print(x.cpu())

I can execute this code on my 14inch macbook pro which provides M1 Pro without rosetta2.

Also, this requires PyTorch 1.11 nightly

Here’s the 1.12 link for Apple silicon gpu backend support link.

My understanding (as day05) illustrates was that cpu training on ARM64 Apple silicon was working already.

Also, this requires PyTorch 1.11 nightly

So, it's not in the stable build, then? We'll definitely wait until it is before picking it up. If it is available in a 1.11 stable build, then we can update the libtorch NuGet packages, and then add the relevant device API logic (device type, index, etc.)

Also, I don't have access to a M1-based Mac... Once we're on 1.12, I may have to ask someone in the community to do the integration.

I mainly use windows but I have access to an M1 Mac and can help test.

Also, I think that it is in 1.11 stable. My understanding from PyTorch forums is that the version number distinguishes stable from nightly. Stable is clean "1.11.0", nightly is something like "1.11.0.dev20201109".

This is what I get after installing 1.11 on an m1 Mac:

> pip install torch==1.11.0`
Collecting torch==1.11.0
  Downloading torch-1.11.0-cp39-none-macosx_11_0_arm64.whl (43.1 MB)
     |████████████████████████████████| 43.1 MB 40.3 MB/s

Then in Python:

Python 3.9.12 (main, Apr  5 2022, 01:52:34) 
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> x = torch.rand(5,3)
>>> print(x)
tensor([[0.2262, 0.6853, 0.9713],
        [0.1158, 0.4903, 0.1279],
        [0.4842, 0.3445, 0.8207],
        [0.4095, 0.3618, 0.7080],
        [0.0212, 0.3360, 0.3603]])
>>> print(torch.__version__)
1.11.0
>>>

For what it's worth, this is what happens when I try to load LibTorch manually on M1:

> open System.Runtime.InteropServices

> NativeLibrary.Load(@"/Users/xxxx/opt/miniconda3/lib/python3.9/site-packages/torch/lib/libtorch.dylib")
val it: nativeint = 8618041536n

> #r "nuget: TorchSharp"
> open TorchSharp
> let lin1 = torch.nn.Linear(1000, 100)

Binding session to '/Users/xxx/.nuget/packages/torchsharp/0.97.0/lib/netcoreapp3.1/TorchSharp.dll'...

error FS0193: Could not load file or assembly 'TorchSharp, Version=0.97.0.0, Culture=neutral, PublicKeyToken=67b0e5e1da516a1c'.

error FS0193: Could not load file or assembly 'TorchSharp, Version=0.97.0.0, Culture=neutral, PublicKeyToken=67b0e5e1da516a1c, processorArchitecture=AMD64'. The given assembly name or codebase was invalid. (0x80131047)

For what it's worth, this is what happens when I try to load LibTorch manually on M1:

> open System.Runtime.InteropServices

> NativeLibrary.Load(@"/Users/xxxx/opt/miniconda3/lib/python3.9/site-packages/torch/lib/libtorch.dylib")
val it: nativeint = 8618041536n

For what it's worth, I don't think you can point at a conda distribution's native libs. If you are going to replace LibTorch, I believe you have to download and unzip the C/C++ distribution.

Also, I've started work on moving on to 1.12 binaries. Running into some problems, though. Hopefully, we can be on 1.12 next week some time.

We will skip 1.12 compat and integrate with 1.13 when that comes out later. That should give us M1 support, too.

Sounds good, thank you Niklas.

@NiklasGustafsson

@NiklasGustafsson would that give us ARM64 on linux too? I see that's mentioned here: https://pytorch.org/blog/prototype-features-now-available-apis-for-hardware-accelerated-mobile-and-arm64-builds/#arm64-builds-for-linux

@nhirschey

@nhirschey, @ericstj, @dayo05

Now that 1.13 is out, we should be getting closer to additional platform support. I will release the first version with 1.13 support as 0.99.0, but not do anything to add support for M1. I'll have to ask someone who has access to a machine with M1 support to work on that.

Reminder: I've just tried to compile libtorchsharp, I got cpuid.h is only exists on x86 system. Maybe we should resolve this

@NiklasGustafsson

#837 Will be help @NiklasGustafsson

Is there any update about this issue?

Addressed in release v0.102.0

Thanks for adding this! Did this also enable the Metal backend (mps) or simply arm CPU?

https://pytorch.org/docs/stable/notes/mps.html

It enables the 'MPS' backend, too.

See:

TorchSharp/src/TorchSharp/Tensor/Tensor.cs

Line 787 in b6e4706

public Tensor mps(bool non_blocking = false)

@NiklasGustafsson

Awesome! Thank you very much @NiklasGustafsson !

@NiklasGustafsson

@NiklasGustafsson These torch.mps apis are not implemented, right? https://pytorch.org/docs/stable/mps.html

I'm looking for synchronize and profiler.*.

Just to be clear, do I need to do anything specific to make TorchSharp use Metal on Apple Silicon? It's not clear to me if TorchSharp-cpu uses Metal.

@derekchiang

@derekchiang I haven't tested this in a while, bug if I recall correctly, TorchSharp-cpu was enough for Metal (unlike CUDA on Windows). You should be able to use some property to query whether mps mode is enabled or not or simply request 'mps' as the device type and see if you get an error or not.

for reference, from an old project, I had:

<PackageReference Include="TorchSharp" Version="0.103.1" />
<PackageReference Include="TorchSharp-cpu" Version="0.103.1" />
<PackageReference Include="TorchSharp.PyBridge" Version="1.4.3" />

With that being said, I never got a performance boost when running on mps. I think something was/is still missing to get that working properly. In other words, I was able to request and run on mps without errors, but it was way slower than using cpu (at least for me).

I contributed some PR that improved cpu performance on macs. At the time I thought the improvement was coming from mps, but it was solely from cpu after all... :(

Navigation Menu

Repository navigation

Apple Silicon support #652

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Apple Silicon support #652

Description

Footnotes

Activity

dayo05 commented on Jul 13, 2022

NiklasGustafsson commented on Jul 20, 2022

dayo05 commented on Jul 20, 2022

nhirschey commented on Jul 20, 2022

NiklasGustafsson commented on Jul 20, 2022

NiklasGustafsson commented on Jul 20, 2022

nhirschey commented on Jul 27, 2022

nhirschey commented on Jul 27, 2022

NiklasGustafsson commented on Jul 27, 2022

NiklasGustafsson commented on Jul 27, 2022

NiklasGustafsson commented on Sep 22, 2022

nhirschey commented on Sep 22, 2022

ericstj commented on Oct 7, 2022

NiklasGustafsson commented on Nov 3, 2022

dayo05 commented on Nov 4, 2022

dayo05 commented on Nov 9, 2022

tugbargn commented on May 4, 2023

NiklasGustafsson commented on Feb 27, 2024

ejhg commented on Jul 9, 2024

NiklasGustafsson commented on Jul 9, 2024

NiklasGustafsson commented on Jul 9, 2024

ejhg commented on Jul 14, 2024

ejhg commented on Jul 27, 2024

derekchiang commented on Sep 1, 2025

ejhg commented on Sep 1, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions