Repository navigation
F# compilation very slow with FCS v41.0.1 #2605
Description
I didn't notice any difference when compiling the tests, but after switching the custom FCS build with the v41.0.1 Nuget package and testing it with a big project (1275 files and I believe it heavily uses SRTP) I realized the F# compilation is very slow, it takes almost 2 times as long as with current Fable version (3.4.10, note that Fable with FCS Nuget hasn't been yet released:
Fable 3.4.10
F# compilation finished in 65379ms
Fable compilation finished in 15852ms
main branch (actually queue-watch)
F# compilation finished in 102581ms
Fable compilation finished in 17020ms
The F# compilation time only measures the ParseAndCheckProject call so the performance decrease must come from FCS, unless I'm initializing the options or the checker in a wrong way.
@ncave I guess it will be difficult to say with so little information, but do you have any idea where the problem may come from? Latest main is built with dotnet sdk 6 but targets net5.0 (I did try to target net6.0 but there was no difference). I remember you pruned off some features in service_slim but don't remember if this also impacted performance or only memory. Should I try to create a repro and report the issue in the dotnet/fsharp repo (problem is the project mentioned above is not public)?
Activity
ncave commentedon Nov 11, 2021
@alfonsogarciacaro Do you see any difference compiling FCS-Fable itself? If not, then it could be some difference for certain F# constructs, in recent FCS package, perhaps you can try older versions of the FCS package to see if that's true.
alfonsogarciacaro commentedon Nov 12, 2021
@ncave You were right, the performance decrease seems to have happened already in a previous release, specifically in v40.0.1-preview. I also tried building standalone but I keep getting this error even with FCS v40.0, which maybe related to the hack you had to make in your fork for Witness issues. It's weird because I think at one point it was working but maybe I made a mistake when testing.
At one point I also tried to create a custom build with latest dotnet/fsharp/main, but I'm getting errors about packages missing from custom Nuget feeds. I missed this brief moment when it was very easy to build FSC :)
We probably should defer the F# 6 upgrade until we can investigate this more, so going back to your fork for now :)
I've reported this in dotnet/fsharp repo: dotnet/fsharp#12380
ncave commentedon Nov 12, 2021
@alfonsogarciacaro Can you use v40.0 for now?
alfonsogarciacaro commentedon Nov 12, 2021
@ncave I was trying to reenable standalone again, but now I'm getting some errors related to how I'm dealing with inline functions now :) I'm trying to collect now the inline info in advance same as we do with entities. I should fix those anyways.
Interestingly, when using your fork and the "standard" FCS API I still get worse performance (this commit):
So there's benefit in keeping the service_slim, but I need some fixes (like adding SourceIdentifier here) so now I've to make a fork of your fork :)
ncave commentedon Nov 12, 2021
@alfonsogarciacaro Sounds like you're saying there may be still some benefits from having a facade on top of FCS. I'll try to find some time to look at updating the fork.
alfonsogarciacaro commentedon Nov 12, 2021
@ncave Seems so. I had some success in rebasing
service_slimwith dotnet/fsharp up to 2021-06-08 (which contains this fix). I managed to build FCS by using this command and I also tried adding alastFileargument toParseAndCheckProject. I was hoping this would avoid the current worst case in watch compilation in Fable (touching a file up in the list triggers checking for all files below even if it's a leaf node) but I'm not getting the expected results (now touching the last file takes much longer than before). Still investigating :/ncave commentedon Nov 12, 2021
@alfonsogarciacaro I thought watch is now using
ParseAndCheckFileInProject, no?dsyme commentedon Nov 12, 2021
If we can get any profiling information for comparison that would be great.
alfonsogarciacaro commentedon Nov 12, 2021
@ncave Yes, but at the end
ParseAndCheckFileInProjectchecks all files before the updated one, and because I know the bulk of changes in each watch iteration (the changed files plus the ones depending on them from Fable's dependency tree), it's easier (and I assume it saves work) if I just tell FCS until what file it needs to recompile.Thanks a lot for checking this @dsyme! I replied in the dotnet/fsharp issue 👍
alfonsogarciacaro commentedon Nov 12, 2021
Not entirely related, but some interesting data. With current
mainI get these timings:Compiling standalone (FCS + Fable itself)
Compiling big project (1275 files)
So with standalone Fable compilation takes longer but with the big project is much faster (this is often the case). I wonder what's causing this difference, but I think this big project uses SRTP extensively (FCS resolves these for Fable now, so that would explain why it takes longer).
UPDATE: Another possibility is that files in FCS+Fable are very big (in they other big project they're much smaller in general) so the parallelization in Fable compilation is less effective.
ncave commentedon Nov 12, 2021
@alfonsogarciacaro Can you post the same numbers with Fable 3.4.10, for comparison?
alfonsogarciacaro commentedon Nov 12, 2021
@ncave Here you go:
Compiling standalone (FCS + Fable itself) with Fable 3.4.10
Compiling big project (1275 files) with Fable 3.4.10
They're very similar which makes sense because at the end
mainis using a version of FCS very close to 3.4.10. Also, as you can see here, FCS v40.0 from Nuget is still a bit slower thatservice_slim(I've updated the comment to include standalone timing).alfonsogarciacaro commentedon Nov 14, 2021
Some interesting things happening. I managed to update ncave's
service_slimto latest dotnet/fsharp. See #2608I did the update in steps trying to find the point when performance dropped, but I couldn't find it. It's a bit slower with latest code which seems to correspond to F# 6 doing more thing but the difference is much smaller that when using FCS API from v41.0 from Nuget. Some timings:
So it looks like the performance decrease doesn't come from the compiler itself but from some bottleneck in service.fs. I'll try to investigate but at least the performance with service_slim is good enough to update Fable to F# 6 👍
alfonsogarciacaro commentedon Nov 15, 2021
Scratch the part about parallelizing the parse phase. This can be done in the first compilation, but after that it messes up with cache invalidation so I've removed it for now.
MangelMaxime commentedon Jul 1, 2026
Doing some clean up