Repository navigation
.NET 6+ WebSocket HttpListener - memory leak #101022
Description
Description
The same server build works differently on Windows and Linux, on Windows maximum memory usage is ~300 MB, then after GC calls it's drop to 180-250 average. On Linux after 5 minutes RAM usage is ~1.3 GB (and grows)
Reproduction Steps
usings:
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.WebSockets;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;Server:
internal class Program
{
static async Task Main(string[] args)
{
var listener = new HttpListener();
listener.Prefixes.Add("http://localhost:7123/");
listener.Start();
while (true)
{
var context = await listener.GetContextAsync();
if (context.Request.IsWebSocketRequest)
{
_ = Task.Run(() => ProcessWebSocketRequest(context));
}
else
{
context.Response.StatusCode = 400;
context.Response.Close();
}
}
}
static async Task ProcessWebSocketRequest(HttpListenerContext context)
{
var websocketContext = await context.AcceptWebSocketAsync(null);
var websocket = websocketContext.WebSocket;
var buffer = new byte[256].AsMemory();
while (websocket.State == WebSocketState.Open)
{
try
{
var result = await websocket.ReceiveAsync(buffer, CancellationToken.None);
if (result.MessageType == WebSocketMessageType.Text)
{
string message = Encoding.UTF8.GetString(buffer.Span.Slice(0, result.Count));
var jsonData = new Dictionary<string, object>();
jsonData.Add("type", "response");
jsonData.Add("body", message);
string serialized = JsonSerializer.Serialize(jsonData);
var responseBuffer = Encoding.UTF8.GetBytes(serialized);
await websocket.SendAsync(responseBuffer, WebSocketMessageType.Text, true, CancellationToken.None);
}
else if (result.MessageType == WebSocketMessageType.Close)
{
await websocket.CloseAsync(WebSocketCloseStatus.NormalClosure, null, CancellationToken.None);
break;
}
}
catch (Exception ex)
{
//Console.WriteLine(ex.Message);
}
}
websocket.Dispose();
}
}Client(s):
internal class Program
{
private static Random s_random = new Random();
static async Task Main(string[] args)
{
Task[] tasks = new Task[2000];
for (int i = 0; i < tasks.Length;i++)
{
tasks[i] = Task.Run(Tick);
}
Console.Read();
}
static async Task Tick()
{
while(true)
{
var clientWebSocket = new ClientWebSocket();
clientWebSocket.Options.SetRequestHeader("Accept-Encoding", "en");
clientWebSocket.Options.SetRequestHeader("Pragma", "no-cache");
clientWebSocket.Options.SetRequestHeader("User-Agent", "MyUserAgent");
clientWebSocket.Options.KeepAliveInterval = TimeSpan.FromHours(20);
clientWebSocket.Options.Cookies = new CookieContainer();
await clientWebSocket.ConnectAsync(new Uri("ws://localhost:7123/"), CancellationToken.None);
int i = 0;
while (clientWebSocket.State == WebSocketState.Open && i++ < 4)
{
var jsondata = new Dictionary<string, object>();
jsondata.Add("type", "request");
byte[] rndBytes = new byte[16];
s_random.NextBytes(rndBytes);
jsondata.Add("body", new Guid(rndBytes).ToString());
string serialized = JsonSerializer.Serialize(jsondata);
await clientWebSocket.SendAsync(Encoding.UTF8.GetBytes(serialized), WebSocketMessageType.Text, true, CancellationToken.None);
var rcvBuffer = new byte[256].AsMemory();
var bytesReaded = await clientWebSocket.ReceiveAsync(rcvBuffer, CancellationToken.None);
string message = Encoding.UTF8.GetString(rcvBuffer.Span.Slice(0, bytesReaded.Count));
await Task.Delay(1000);
}
await clientWebSocket.CloseAsync(WebSocketCloseStatus.NormalClosure, null, CancellationToken.None);
clientWebSocket.Dispose();
}
}
}Expected behavior
Same memory usage as on Windows
Actual behavior
Memory leak?
Regression?
Same on .NET 8 ( Microsoft.NETCore.App 8.0.4 )
Known Workarounds
No response
Configuration
Windows:
Windows 10 x64, Version 10.0.19045 Build 19045, 22H2
Microsoft.NETCore.App 6.0.27
Linux:
Debian 11 (5.10.0-28-amd64 Debian 5.10.209-2 (2024-01-31) x86_64 GNU/Linux)
Microsoft.NETCore.App 6.0.27
Other information
No response
Activity
dotnet-policy-service commentedon Apr 14, 2024
Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.
janvorli commentedon Apr 17, 2024
@zxcvqwerasdf thank you for sharing the source code for the repro! I will give it a try. Did both of the windows and Linux machines you were using have similar specs (memory, number of CPU cores)?
zxcvqwerasdf commentedon Apr 17, 2024
One is vps kvm 2 cores 4 ram, second is vps kvm 1 core 1 ram, third is virtual machine in virtualbox 8 core 16 ram. Same behavior on all specs.
janvorli commentedon Apr 17, 2024
I was able to repro the memory growth on Linux with .NET 9 too (haven't tested windows). I've taken a dump after the memory consumption reached about 1.2GB. There is some managed leak, just look at the list of the GC heap objects with the largest counts - over 46000 live HttpClient instances doesn't look healthy.
Here is a summary of the memory usage, you can see that the GC heap is 830MB large:
karelz commentedon Jun 25, 2024
@CarnaViire will you be able to take a look at it?
rokonec commentedon Jul 23, 2024
@zxcvqwerasdf We have identified the issue and found workaround. We are considering a fix for this issue but we are unsure when it will be actually fixed.
Meanwhile please add
context.Response.Abort();in server right afterwebsocket.Dispose();so it looks like this:This code will work with both Windows and Linux.
We apologies for inconveniences caused.
2 remaining items
janvorli commentedon Jul 23, 2024
cc: @mangod9 - this is a memory leak reported couple of months ago where the culprit was just figured out.
mangod9 commentedon Jul 23, 2024
Ah thanks for the FYI. Is this 6 only or occurs in 8 too?
janvorli commentedon Jul 23, 2024
I believe .NET 9 too, as I have reported above that I could repro it in 9.
rokonec commentedon Jul 23, 2024
@mangod9 I believe it is 6+. I have reproduced it with 9 main.
karelz commentedon Jul 25, 2024
Triage: This is problem in HttpListener, not in ClientWebSockets. It is rarely used, first report in years, moving to Future.
Workaround: Abort the response - see above.
[-].NET 6 Linux ClientWebSocket memory leak[/-][+].NET 6+ WebSocket HttpListener - memory leak[/+]