Repository navigation
Native aot'ed shared libraries should not be unloadable on unix #64629
Description
Coreclr/CoreRT are not designed to be unloaded without process exit.
In case of anative aot'ed shared library, it's possible it will be used in dynamic loading/plugin scenarios, so it might be loaded and then possibly unloaded by the plugin host, causing process to malfunction/crash.
This has been fixed for windows (commit 7b6c0025482aaf8476a6f4ecd357e89a0516ce47) by pinning the library in memory so that it cannot be unloaded, however there is no such fix for unix.
This is still marked as a TODO in file https://github.com/dotnet/runtime/blob/main/src/tests/nativeaot/SmokeTests/SharedLibrary/SharedLibrary.cpp#L79
Activity
webczat commentedon Feb 1, 2022
cc @MichalStrehovsky If this is not a dupe I would be happy to fix it.
MichalStrehovsky commentedon Feb 2, 2022
I've looked a bit more into this with the suggestion from dotnet/corert#7011 (comment).
If we want to use dlopen with RTLD_NODELETE, we need to get a path to the library first. We would probably want to do all of it around here:
runtime/src/coreclr/nativeaot/Bootstrap/main.cpp
Lines 171 to 174 in b2dc37b
webczat commentedon Feb 2, 2022
the commit mentioned above does this in a function named PalGetModuleHandleFromPointer. I have looked at the unix equivalent of this function and it uses
dladdrto fetch the handle, and the same function also gives the library pathname.Although doing it there looks a bit out of place to me, we have a precedent of using dladdr in the codebase for unix context, so I could either use dladdr in the place you mentioned or possibly just dlopen/dlclose when the above mentioned function is called. dlclose is needed so that refcount doesn't increase needlessly.
The only thing I don't know is whether OSx allows what linux does, adding a nodelete flag to an already loaded library, which I tested works on linux.
webczat commentedon Feb 3, 2022
I know it's not directly related to the issue but before trying to fix things in any way, I anted to first have a test case showing the fix was effective.
I have tried to uncomment the dlclose line in src/tests/nativeaot/SmokeTests/SharedLibrary/SharedLibrary.cpp but it seems like on linux/glibc the library does not get unloaded, even though it does not have nodelete flag. I have no idea why this is happening so I wouldn't rely on that as it might be specific to linux or glibc, or might even be a coincidence. I have also verified that I can still call the library functions after dlclose.
I know that in case of a plain program not linked to anything except standard libs and libdl, during loading/dlopening the native aoted shared library, what is also loaded is librt, libpthread and then dlopened libicu. rt and pthread are marked nodelete during loading, but I don't see any traces of SharedLibrary.so being marked nodelete.
When searching for the possible problem causes one can find https://stackoverflow.com/questions/38869657/dlclose-not-unloading-so-file-which-is-linking-to-boost but IMO this might not be the same problem as, if I am not mistaken, the text of the top answer suggests that the dlopened library should be marked nodelete automatically in this case, yet it seems not happening here.
I have tried to reproduce things by creating a program dlopening a library which dlopens (and leaves opened) another library. Then the first library is closed and gets unloaded even when I also link it with libpthread and librt, which I confirm are loaded too.
Any clue what may cause this? I doubt it's just the corert suddenly not being unloadable.
MichalStrehovsky commentedon Feb 6, 2022
I guess it's good news that it won't unload, but I don't have a good idea why either.
webczat commentedon Feb 7, 2022
I don't thing it's good because we don't know why and that condition might suddenly become false, or be false on musl or on mac or even on a different compiler. But I have confirmed that the library cannot be unloaded via repeatedly dlclosing it.
DeafMan1983 commentedon Apr 30, 2023
I am sorry @MichalStrehovsky You mean library can't unload. Is it possible
dlcloseor other method under unix-like functions. Or possible Unload, Reload....webczat commentedon Apr 30, 2023
the intention is they are not unloadable at all, also with dlclose.
MichalStrehovsky commentedon May 1, 2023
Yup, unloading them at this point is equivalent to pulling the rug from under it. There will still be treads running various things and their code/static data just got unloaded. Nothing good comes out of that. Even if it doesn't crash, we leaked a bunch of memory and handles.
ulrichb commentedon Feb 4, 2026
We have the situation that our Linux .NET AOT UnmanagedCallersOnly shared library is unloaded by the calling process (it's a plugin model) which we don't have under our control. This ultimately leads to segfaults during OpenSSL cleanup (#123140) after the .NET shared library has already been unloaded.
As a workaround, we want to prevent that our .NET module from being unloaded. Currently we're using this approach:
dladdr()to get thedli_fnameof our own shared library (full path of the.sofile).dlopen()it during the static c'tor of the UnmanagedCallersOnly-entry point and leak (never close) it on purpose.I.e. we use the re-using of the "same object handle" + reference counting behavior of
dlopen().This works in our scenario. However, I'm unsure whether that's the best workaround. Do you have suggestions for better ones? Do you see situations in which this solution is problematic (e.g. I'm thinking of situations where the re-using of the object doesn't work, thus creating an endless loop)?
webczat commentedon Feb 4, 2026
I remember myself wanting to fix that but trying to reproduce but somehow I couldn't at the time, somehow trying to dlclose didn't unload in my tests.
There is a nodelete flag that a library can be opened with or compiled with to prevent unloading that I wanted to use as a fix on linux. Then you literally cannot unload via dlclose.
josephmoresena commentedon Feb 4, 2026
Where do you do this: in the calling process or inside the shared library? If it’s inside the library, can you use a ModuleInitializerAttribute marked method?
am11 commentedon Aug 2, 2026
I was experimenting with replacing RapidJSON in corehost with STJ (for #131478) and ran into the issue that dlclose doesn't cleanup everything initialized by AOT runtime. One approach is that runtime provides a teardown API such as
shutdown_runtime, so people can dlsym it and call it before dlclose the lib. Something for the future perhaps?As it stands, we can use the sandbox approach; fork() on unix and single binary on windows, with something like https://github.com/Snaipe/BoxFort. It allows asynchronous and multi threading comms between parent and child. Here is a basic / stripped version of sandbox for a single thread with a sample glue:
sandbox_runner.tar.gz (
./run.shto build and run)fork -> starts fd listener -> dlopen -> dlsym call(s) -> dlclose this makes the OS cleans up the sub process with all its threads, signal registrations and memory allocations when we are done
sandbox_destroy(Sandbox*).Ensure that Native AOT libraries are not unloaded (#131960)
Ensure that Native AOT libraries are not unloaded (dotnet#131960)