Dext on Windows: a socket engine with what HTTP.sys gives, and faster

[Stemonik]

Hi Cesar,

The PRs I opened (#279, #280, #281, #282) come from one question: where does the fixed cost of a request go on Windows. The answer is mostly HTTP.sys itself. About 1.8 context switches per request happen in the kernel's own threads (System process), on top of ours; the same .NET application on HTTP.sys instead of Kestrel loses 35-75% on the same machine, so this is the price of HTTP.sys, not of Dext (Dext uses it better than Microsoft's own server does).

The IOCP socket engine that Dext already has does not pay that price. With #282, and a few lines in TDextNativeWebServer to pick it on Windows (an environment switch in our bench), same machine, same binary:

50 connections, req/s	HTTP.sys (with the fixes)	socket engine
Select 1 record, no auth	31,370	35,302 (+13%)
Select 1 record, JWT	31,476	35,055 (+11%)
Select 50 records, no auth	17,232	17,719 (+3%)
Select 50 records, JWT	16,157	17,788 (+10%)
Ping, no auth	95,449	91,568 (-4%)
Ping, JWT	94,876	101,540 (+7%)
Insert 100 records (bound by the PostgreSQL commit)	2,299 / 2,384	2,308 / 2,234 (noise)

CPU per request on Select 1: 108 µs with HTTP.sys, 91 with sockets, almost all of the difference in the kernel (57 -> 42 µs). Against the other servers in the same run (Rust/axum, Go, .NET 10/Kestrel, Node, Spring Boot), Dext with the socket engine has the most requests per second on Select 1 and Select 50; the nearest, Rust, is 4-6% behind without auth. Everything, with the method and the raw data, is here: https://github.com/Stemonik/dext-benchmarks

What HTTP.sys gives today that the socket engine does not

• TLS, with certificates from the Windows store and SNI bindings (IsSecure is False in the socket engine: "Phase 1/3 does not implement SSL yet");
• timeouts and limits in the kernel (WithHttpSysTimeouts, WithQueueLength, WithMaxBandwidth): a slow client cannot hold a connection forever;
• one port shared by URL prefix with other services and IIS;
• HTTP/2 under TLS, negotiated by HTTP.sys itself;
• and the maturity of a component that has faced the internet for over twenty years.

The invitation

I think Dext on Windows deserves a server engine with these features and the speed of the socket engine. Two roads, and the choice is yours:

1. complete the socket engine: TLS (on Windows SChannel through SSPI would avoid shipping OpenSSL), timeouts (a periodic sweep of the sessions), chunked request bodies, IPv6 and SO_EXCLUSIVEADDRUSE, then an option on the server builder to choose it. This is the road Kestrel took on Windows: a user-mode socket transport, TLS through SChannel;
2. another design you prefer that keeps what HTTP.sys gives and drops what it costs.

HTTP.sys would stay the right default for port sharing by prefix and for whoever wants the kernel to do the TLS.

If you choose the first road, I can take the pieces that need no design decision as separate PRs, with tests like those of #282: timeouts, chunked bodies, IPv6 and SO_EXCLUSIVEADDRUSE, the option to choose the engine. TLS I would leave to you, or do it on a design you give me.

P.S. One more small thing, measured separately: mimalloc (and other per-thread allocators) can trim a thread's heap only from that thread. A hook called once on a worker that handled work and then waited one second without any (DextWorkerIdle, in the HTTP.sys and socket worker loops) lets the application do it: after a load it halves the private memory at rest, with no measurable cost under load. I can send it as a PR if you like the idea.

Best regards,
Stefano

[cesarliws]

Thanks, @Stemonik. The measurements, the way you split the work, and the tests
in #282 are exactly what I hoped to see from a contribution. I'm going through
#281 and #282 separately and will comment there.

I'm in favour of the first road, step by step, and a good part of what you
offer already has a place in our specs:

• Timeouts: S50 section 3.6 describes a timing wheel per worker for the epoll
  engine. The IOCP engine has no equivalent yet, so mirroring that design would
  be a natural first step. It would also close the case of a connection left
  half-closed after BeginClose.
• Chunked request bodies and keep-alive pipelining are listed in the IOCP
  phase of S39.
• IPv6, SO_EXCLUSIVEADDRUSE and the option to choose the engine: yes, please.

I just published a few specs that are related, so you can see what we already
have: S39 now has the IOCP status and a section of open decisions (section 8),
and S43 (native TLS) has its Status and Validation Plan next to it:

• https://github.com/dotpas/dext/blob/main/Docs/Specs/S39-Native-Server-Engine.md
• https://github.com/dotpas/dext/blob/main/Docs/Specs/S43-Net-Advanced.md
• https://github.com/dotpas/dext/blob/main/Docs/Specs/S50-Linux-Epoll-Evolution.md

About how it would be offered: I'd like the socket engine to be one more server
provider, next to the ones we have today (Indy, DCS, WebBroker and the native
http.sys/epoll one), so each developer picks what suits them, and I'd leave
http.sys as it is. Today TDextNativeWebServer fixes the engine per platform. A
way in could be to let it receive the engine, and to give the IOCP engine a
Factory like the one http.sys has, so it is chosen with UseServerFactory. Does
that fit what you had in mind? In the docs I'd call it experimental until
timeouts and TLS are in, with a note that direct exposure to the internet needs
a provider that has TLS today (http.sys, or Indy with Taurus) or a
TLS-terminating proxy. It would also help if our integration and middleware
tests could run against every provider we offer.

On TLS, I'd love to work through the design together, and we are not starting
from zero. Dext already has a TLS abstraction (S43: IDextTLSEngine,
IDextTLSContextProvider, IDextTLSStream) with an OpenSSL Memory BIO engine, and
the epoll engine already uses it per connection. The IOCP engine is the one that
does not use it yet (IsSecure is still False).

There is an open question where your view would help a lot: on Windows, should
the socket engine offer SChannel (no extra DLL, Windows certificate store, which
fits the "no DLL beyond the system ones" goal of S39), OpenSSL (already written,
same behaviour as Linux), or both behind IDextTLSEngine? I lean towards both
behind the interface, but I'd like to hear what you found while measuring. If
you'd like to start on TLS, a first step that fits well is plugging the IOCP
session into IDextTLSEngine the way Dext.Server.Epoll does, with OpenSSL. We can
then see together where an SChannel provider fits.

I'll reproduce the benchmark on my machine, so we can quote the numbers in the
docs with confidence.

About DextWorkerIdle: yes, please send the PR. A hook that does nothing by
default and costs nothing in the worker loop when nothing is registered is easy
to accept.