Building the .NET Core SDK on FreeBSD

I'm trying to build the .NET Core SDK on FreeBSD but I'm having trouble with the documentation.
The documentation says I just have to initialize the submodules in the source-build repo and run ./build.sh. But this produces only the following output:

Unsupported OS 'FreeBSD' detected. Downloading linux-x64 tools.
Installing dotnet cli...
Installing dotnet 3.0 preview1 cli...
Restoring BuildTools version 3.0.0-preview4-04104-01...

Later in the documentation it says that

git submodule update --init --recursive [...] will initialize the submodules and clone the repos under the src folder if they haven't been initialized.

But after running this command, my src folder contains only the subfolders netcorecli-fsc and reference-assemblies, which are not all required repos I think. As the documentation says, I expected at least core-setup, standard, corefx and coreclr to be cloned into the src folder.
This would also explain why there is no bin folder in my source-build repo after running ./build.sh as described in the documentation.
If the scripts don't work under FreeBSD, I expect at least an error, otherwise I assume that everything has been built, which is not the case here.

So, am I doing something wrong or is the documentation incorrect or out of date?

even source-build needs working SDK and that is problematic since lost build servers. I was planning to revisit the effort but closing on 3.0 prevented me from spending any reasonable time on it.

Aside from that, more cleanup may be needed to deal with OS differences.

I've already tried to use the old (December 2018) but working .NET Core SDK for FreeBSD as bootstrap CLI as described here. But when running ./init-tools.sh it produces the same output as above and doesn't finish even after several hours.

I could spend some time trying to re-enable .NET Core for FreeBSD, but I need to know where to start because the current documentations don't help me much if they don't work. For example, it would be helpful to know why the build servers were lost and how the .NET Core SDK for FreeBSD that I mentioned was created at that time.

That is something source-build gurus should comment on. Published SDK was built by bootstrapping each needed repo separately and by publishing and consuming intermediate packages. As more repos moved to Arcade, lot of the build process was in flux. I would not be surprised in documentation or code is out of sync.

We've changed our process so that repos are cloned as part of the build process to ./bin/src. This lets us piggyback off other .NET infrastructure to keep source-build up to date more easily. netcorecli-fsc and reference-assemblies are both fake repos that live in source-build, so it's expected you'll have those in ./src. Your error from ./build.sh looks like init-tools is failing - could you post your init-tools.log from the root of the repo?

Yes, init-tools is indeed failing. The first error was a problem with the SSL certificate while restoring NuGet packages. I could fix it by setting the SSL_CERT_FILE environment variable to /usr/local/share/certs/ca-root-nss.crt as described here. But now there seems to be another problem with the SSL connection:
init-tools.log
The SSL certificate from https://dotnet.myget.org/F/dotnet-buildtools/api/v3/index.json exists in my ca-root-nss.crt file. So I'm surprised that NuGet package restore still seems to have problems establishing an SSL connection.

It looks like dotnet has problems with the linux-c7-openssl-libs package. I installed it on my FreeBSD machine to ensure full Linux binary compatibility, because the first time I ran ./build.sh I got
No usable version of libssl was found.

After removing this package, reinstalling the default OpenSSL package for FreeBSD, and creating the required symbolic links, running ./init-tools.sh using the .NET Core SDK for FreeBSD as bootstrap CLI succeeded.
I can now tell build.sh to use .NET Core SDK version 3.0.100-preview-010021 via editing DotnetCLIVersion.txt, but running it still fails:
error.txt

Ah, that looks like Darc (the tool we use to clone the source) doesn't build for BSD. I wonder if we can build that portable, I will take a look. Otherwise this is probably a fix in dotnet/arcade-services.

I was able to build Darc on FreeBSD using target framework netcoreapp3.0 instead of netcoreapp2.1. Cloning the repos now seems to work, but building them still fails. I have run ./build.sh and received the following output:
arcade-error.txt
arcade-build.binlog.txt
arcade.log

I also ran ./build.sh /p:RootRepo=core-setup and received the following output:
core-setup-error.txt
core-setup-build.binlog.txt
linker.log

So in both cases I get the error Segmentation fault (core dumped).
I also receive linux: pid 1234 (dotnet): syscall membarrier not implemented in the terminal, which is not logged into the files.

I've seen similar errors with missing syscalls on ARM. At this point it's probably less of a source-build problem and more getting into bootstrapping the actual SDK on BSD - there's a couple related issues: https://github.com/dotnet/core-setup/issues/5083, dotnet/installer#248, #724.

@crummel

Looks like it is a bootstrapping problem, but probably not as you expected @crummel: Although I specified to use the .NET Core SDK for FreeBSD as bootstrap CLI, the build scripts also downloaded the .NET Core SDK 3.0.100-preview5-011568 for Linux and placed it in the source-build/Tools/dotnetcli3 folder. After replacing the folder with source-build/Tools/dotnetcli, which contains the .NET Core SDK for FreeBSD, the build continues.
Now it tries to build Microsoft.DotNet.Arcade.csproj, but fails with error CS1617: Invalid option 'preview' for /langversion. Use '/langversion:?' to list supported values.

Replacing <langversion>preview</langversion> with <langversion>latest</langversion> in Microsoft.DotNet.Arcade.csproj fixes the above error. It looks like I'm now able to build the main parts of the .NET Core SDK on FreeBSD. I only get the following two errors after the build succeeded:

'common' failed during build.
See '/root/git/dotnet/source-build/bin/logs/common.log' for more information.
'known-good' failed during build.

I've already taken a look at the common.log, but there's nothing there that indicates an error.
So does anyone know what's common and known-good? Are they required for the .NET Core SDK? And where is the .NET Core SDK stored after the build is complete? Or do I have to copy and paste the parts manually?
Any help would be appreciated. Thanks in advance.

AFAIK known-good is the set of components that need to be built to assemble .NET Core itself. Right now, it's roughly analogous to the all target in other projects' makefiles.

If everything (including the cli) built succefully, the result should be a bin/x64/Release/dotnet-sdk-*tar.gz file. That's the .NET Core SDK itself. For fedora that's pretty much all we need. We extract it to /usr/lib64/dotnet.

@joperator

That is great progress @joperator. Do you have any outstanding changes or just something you need to set?
With 3.0 under control now, I should have some cycles again to give FreeBSD another try.
Just to confirm: You use FreeBSD bootstrap cli, not Linux under emulation, right?

@omajid

@omajid: Then it looks like I have to build known-good as well because my bin/x64/Release folder only contains an empty netcorecli-fsc directory.

@wfurt: There are outstanding changes as well as some things you need to set to successfully run build.sh on FreeBSD. But these changes are only limited to a few files, where some of them are already described above. I will soon provide a complete list of these necessary changes.
I'm using FreeBSD bootstrap CLI to build the .NET projects. While running build.sh it downloaded and tried to use the .NET Core SDK for Linux. But that didn't work, as I described above.

@joperator

Very good @joperator. I'm hoping to get newer version of the SDK published (closer to 3.0 final content) but I could not build some components again. If you get build working it would help me with that effort.

As far as the Linux SDK: I made changes in several components to recognize FreeBSD properly. I was on Linux path for a while until I realized that is dead-end path. Even if the binaries work under emulation it still uses many files from /proc and that will not work on FreeBSD. Getting source build working is best long-term option for FreeBSD. I would still like to build components using published sdk to avoid regressions.

@spchamp

@spchamp I am both excited and intrigued by your progress because there is one thing that isn't totally clear to me: what is the goal?

I tend to think that your end goal is to 1) create a FreeBSD port using bhyve behind the scenes to build the SDK, and 2) enable the .NET team to build for FreeBSD more easily upstream by providing them a docker image that they can automate using Github Actions.

If this understanding is correct, for 1) are we sure that the build system of the official FreeBSD package repository will allow us to run bhyve to build the port? There are all sort of limitations there such as the fact that it is not possible to fetch anything after the fetch stage of the build progress.

For 2) my (limited) understanding is that the .NET team uses Azure Pipelines to build .NET so I struggle to understand how the Github Action integration will help them. At the same time since docker is not supported on FreeBSD I assume that this is not intended to be used by FreeBSD users, so the goal of the Docker part of your effort is obscure to me.

Could you please help me clarify the goal that you have in mind

@nkosi23

@nkosi23 although I've been looking at a couple of ideas for how to automate the build outside of my own own VM configuration, I don't believe anything is really concrete about it as yet. Presently, I'm simply trying to test the build under a certain VM configuration using openSUSE. I was pretty glad that the build actually worked out, though.

After some further reading about some ostensible options for server-side automatation in some cloud network, I understand that it may not be possible to build the FreeBSD build dependencies under Docker here at GitHub.

With regards to any ostensible Docker-based orchestration for build automation at GitHub: Although I have an openSUSE installation under Bhyve available for some initial testing, but it seems that the GitHub Workflow documentation favors Debian-based distributions for Docker automation. I'm assuming that that's what they're running on the servers - maybe they're using an Ubuntu release of some version. So, I'm going to test the build under a Debian-based distribution locally, before putting together anything in Docker with GitHub.

That would not serve to address the concern of the missing build-dependency for lttng-ust. Without some kind of a patch for the ports tree, that port might not build for amd64. At last check, it seemed that there is not any pkg available for that port, on amd64, in official FreeBSD port builds at this time. If it may be possible to build a patched port tree for purpose of the build dependencies, of course that would have to be conducted outside of Docker - unless there would be some way to build a FreeBSD base system and a small number of ports for a FreeBSD system, within a Linux compiler environment, lol.

If automating the build-dependencies part in some FreeBSD system, perhaps it could also serve to provide any build options for different GSS-API implementations within the cross-build environment and thus within any installation environment (e.g MIT krb5, heimdal, or krb5 in the base system) and for DTrace support in the build. Coordinating this with some Linux-based system for the cross build may not be posible singularly at GitHub. I'll try to take a look at what I may be able to put together for an independent build server with Azure DevOps or alternately, DigitalOcean, for automating that part of the build. Perhaps it might be too much of a hack to try to build a FreeBSD base system and any number of ports, if within qemu with Docker orchestration at GitHub.

Assuming that the entire build process can be automated for any single set of .NET SDK and framework versions, perhaps it may be possible to automate some method of traversal across different SDK and runtime versions, for building any additional releases with a similar methodology.

Subsequently, perhaps the cross-builds produced under Linux could be used to produce any builds independent of the cross-buld orchestration, on some FreeBSD system - ostensibly independent to anything related to any FreeBSD brand name, other than in the uname aspects, and source and distribution rights.

Assuming then that this will have to be produced entirely independent of anything related to any official FreeBSD project, other than in the source code and legal aspects in the software distribution, then in order to make use of any such unofficial .NET SDK and runtime builds on a FreeBSD system - there are a few more bases to cross, before that final stage in the distribution concern.

For local builds, of course there's Bhyve, or Linux Containers, or the ostensible "bare metal" Linux installation, or even Qemu, furthermore any analogous systems that may be available on other operating systems.

Under one configuration, the buld has worked out with openSUSE under Bhyve.

Candidly, that is about the limit of what I have in mind. Health, all.

@spchamp

To my best estimate with the tree I'm working with locally, the build needs up to 16 GiB of filesystem space.

@spchamp, In case you are not already using it, there is a /p:CleanWhileBuilding=true build option that will reduce the size needed to build.

@sec

So, how is the progress? Did we come any closer to having modern .NET stack in FreeBSD ports? I've noticed that @sec is updating his repo more or less regularly (last commit being this year), are there any connections between his work and upstream?

In FreeBSD ports, we currently rely on Mono to build C# projects, and recently some developers become more feared it's being phased out, sometimes dubbed as "dead upstream". While this is not quite true AFAICT, there's clearly an increasing demand for modern .NET implementation which FreeBSD is unfortunately lacking.

Update: I've discovered a more actively maintained issue with very recent status update. Sorry if it was referenced earlier, I must have missed it in this case.

@danfe

@danfe very close. There are a few issues I noted in https://github.com/dotnet/runtime/issues/14537. There are two matching PRs open to fix the R2R issue. The others need to be opened/fixed here in source-build.

After the above PR's, the build process would look like:

download pre-built SDK
clone dotnet repo
unpack SDK into repo
./prep.sh
manually add missing NuGets (issue noted above)
manually patch FreeBSD RIDs into installer
./build.sh --online

Additional considerations when building:

No official support means manually downloading and unpacking a pre-built SDK (noted above)
- There are three public sources for this: @sec has native built ones for x64 and ARM64 for FreeBSD 13 and both I and the Servarr team have crossbuilt ones for FreeBSD 12 x64
No official support means manually downloading the missing FreeBSD NuGets (noted above)
- Both I and @sec provide NuGet feeds with our prebuilt items
No official support means manually patching RIDs into installer (noted above)
- Intentional. FreeBSD is community supported. We do not want to create a massive support headache for Microsoft if people think there is official support so the RID must be manually patched here
allow.mlock=1 in jail when building/running
AFAIK offline building is still not supported for source-building, which means the ports system can not build this
- This will hopefully be fixed before net8 launches

@danfe

@danfe we now have .net 8 port in FreeBSD ports tree, ready for use (x64 and arm64).

With lang/dotnet in FreeBSD port I think think this is safe to close?

The port uses the VMR with a prebuilt bootstrap and some script work to make sure it works under the constraints of FreeBSD's build and pkg servers.

Those wanting a portable build can always use either a) cross-building from Linux or b) native non-VMR building. The patches from ports should be enough but there are a few PRs for future versions of dotNET that portable builders should either cherry-pick or wait for them to get merged (e.g. FreeBSD RID, OpenSSL3 support)

With lang/dotnet in FreeBSD port I think think this is safe to close?

This FreeBSD port is very good for installing a ready-to-use .NET SDK on FreeBSD (something I've wanted for years). However, maybe I'm a bit traditional, but I'd still like to have some kind of documentation on how to build the .NET SDK on or at least for FreeBSD using native build and/or cross-build on Linux.

@joperator

@joperator the port uses dotnet's VMR to generate a non-portable build. if you have a ports tree setup, you can just make build in lang/dotnet and it will produce all of the items need to not only install the SDK (make install works too) but build SDK minorversion+1 from the VMR.

If you want portable native builds @sec has an excellent set of scripts setup for that and even publishes SDKs with a handful of NuGets

If you want portable cross-builds from Linux I have a repo for that. I publish the SDK along with NuGets too. I do however need to spend some time improving the scripts as the default container switched from ubuntu to cbl-mariner and that breaks a number of things.

Both of us maintain our own NuGet feeds if you just need NuGets.

My general advice is to use the port as pkg install dotnet is easier and quicker than the other options. If you want to build a portable version @sec scripts will be more useful for pure FreeBSD users. My repo will be better for those that want to produce FreeBSD items but work in Linux/OSX

Finally, for official documentation? Well, FreeBSD is still a community supported platform so it only has instructions to build the runtime repo and nothing more. The FreeBSD wiki entry might be a better place but it needs some updates. The only major difference between natively building and cross building from Linux is that the native builds need a working SDK and a handful of NuGets setup before hand while the cross build can download just about anything it needs from Microsoft's feeds. The cross build requires docker unless you want to create your own crossrootfs.

It's not an official documentation, but your comment is something I asked for: A quick overview where to find scripts and instructions to build the .NET SDK on/for FreeBSD. Thank you very much and it's great to see how far the community has come with this. 🎉

Maybe we should put some additional code to the port to ease cross-compiling? Like, some knobs or targets hidden in the Makefile that may be adjusted/ran by the end user.

Navigation Menu

Repository navigation

Building the .NET Core SDK on FreeBSD #1139

201 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Building the .NET Core SDK on FreeBSD #1139

Description

Activity

wfurt commented on Jul 15, 2019

joperator commented on Jul 15, 2019

wfurt commented on Jul 15, 2019

crummel commented on Jul 15, 2019

joperator commented on Jul 16, 2019

joperator commented on Jul 16, 2019

crummel commented on Jul 17, 2019

joperator commented on Jul 18, 2019

crummel commented on Jul 18, 2019

joperator commented on Jul 30, 2019

joperator commented on Jul 31, 2019

omajid commented on Jul 31, 2019

wfurt commented on Jul 31, 2019

joperator commented on Aug 1, 2019

wfurt commented on Aug 1, 2019

201 remaining items

nkosi23 commented on Apr 10, 2022

spchamp commented on Apr 11, 2022

MichaelSimons commented on Apr 11, 2022

danfe commented on Apr 4, 2023

Thefrank commented on Apr 4, 2023

sec commented on Feb 4, 2024

Thefrank commented on Feb 13, 2024

joperator commented on Feb 14, 2024

Thefrank commented on Feb 14, 2024

joperator commented on Feb 14, 2024

arrowd commented on Feb 14, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions