Repository navigation
System.Net.*.Tests are unstable or slow/long-running on a S.P.CoreLib checked runtime #131
Description
Activity
- changed the title
[-]System.Net.*.Tests are inestable or slow/long-running on a S.P.CoreLib checked runtime[/-][+]System.Net.*.Tests are unstable or slow/long-running on a S.P.CoreLib checked runtime[/+]on Nov 19, 2019 Do we know why especially the networking tests are so slow on a checked runtime?
What is a "checked" runtime? I know what Debug vs. Release is. How does "checked" related to that?
Checked is the same as Debug (e.g., asserts enabled), but with the C++ compiler optimization enabled (so native code runs much faster).
It would be great to get more specific instructions how to know produce one and to know if test run is checked. I always assumed that checked would be slower than Debug but it seems like it may not.
Lot of networking code (product & tests) depends on timing. That is different from let say data structure tests. There are some constants for test and AFAIK we already adjusted some for ARM runs. We could possibly do that for checked builds if needed. But first we need to be able to reproduce it local and take a look.Reacted by Karel ZikmundAny update on this? @BruceForstall @ViktorHofer @safern
Here is a Checked run, where this tests had to be killed because they were long running:
2 remaining items
- added a commit that references this issue
on Oct 12, 2020 Due to lack of recent activity, this issue has been marked as a candidate for backlog cleanup. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will undo this process.
This process is part of our issue cleanup automation.
Seeing the same timeout failure despite the test being disabled on non-release runtimes
https://helixr1107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-115234-merge-60c0a58a5d5d44bf9e/System.Net.Mail.Functional.Tests/1/console.f4ccdd97.log?helixlogtype=resultSystem.Net.Mail.Functional.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing Discovering: System.Net.Mail.Functional.Tests (method display = ClassAndMethod, method display options = None) Discovered: System.Net.Mail.Functional.Tests (found 226 test cases) Starting: System.Net.Mail.Functional.Tests (parallel test collections = on [2 threads], stop on fail = off) System.Net.Mail.Functional.Tests: [Long Running Test] 'System.Net.Mail.Tests.SmtpClientAuthTest_SendMailAsync.TestCredentialsCopyInAsyncContext', Elapsed: 00:03:20 System.Net.Mail.Functional.Tests: [Long Running Test] 'System.Net.Mail.Tests.SmtpClientAuthTest_SendMailAsync.TestCredentialsCopyInAsyncContext', Elapsed: 00:05:20 System.Net.Mail.Functional.Tests: [Long Running Test] 'System.Net.Mail.Tests.SmtpClientAuthTest_SendMailAsync.TestCredentialsCopyInAsyncContext', Elapsed: 00:07:20 System.Net.Mail.Functional.Tests: [Long Running Test] 'System.Net.Mail.Tests.SmtpClientAuthTest_SendMailAsync.TestCredentialsCopyInAsyncContext', Elapsed: 00:09:20 System.Net.Mail.Functional.Tests: [Long Running Test] 'System.Net.Mail.Tests.SmtpClientAuthTest_SendMailAsync.TestCredentialsCopyInAsyncContext', Elapsed: 00:11:20 System.Net.Mail.Functional.Tests: [Long Running Test] 'System.Net.Mail.Tests.SmtpClientAuthTest_SendMailAsync.TestCredentialsCopyInAsyncContext', Elapsed: 00:13:20 ... [EXECUTION TIMED OUT]
We're disabling some System.Net.*.Tests assemblies when running on a checked runtime in: #261. However we should investigate what can we do to enable these tests there.
I.e System.Net.Sockets.Tests would be a really would candidate for runtime testing and stress testing.
cc: @BruceForstall @jkotas @stephentoub