Network recieve DDP performance

[ssjgesus]

Running on a p4 but seems to occur on an s3 as well.
Even with no led drivers, network recieve running over eth, sender being either wled or ledfx. Once the led count gets over a few thousand, the ddp display gets very stuttery - best seen with effects that use black space. Ie matrixpix or similar.

The same ddp sent to wled plays smoothly.

[ewowi]

Reproduced, found a real cause, and fixed part of it. Thanks for the clear report, the "even with no led drivers" detail is what pointed at the right place.

What was wrong

NetworkReceiveEffect keeps a staging buffer (hold-last-frame) and copied it into the layer buffer every tick, whether or not a packet had arrived. The Layer does not clear its buffer between frames, so that copy was writing identical bytes over themselves. At 12288 lights that is a 36 KB memcpy per tick for nothing.

Guarding the copy on "did a packet actually land" cut the receive effect's idle cost from 3522 us to ~240 us per tick on an ESP32-S3, a 93% reduction. No frames are dropped and hold-last-frame is unchanged, the copy is skipped only when nothing new arrived, in which case the pixels are already there.

Measured, before and after

Bench: ESP32-S3 over WiFi, 12288-light grid, no LED driver attached. We used HTTP availability (6 requests during a send) as the yardstick, since it is objective and it degrades in step with the stutter.

lights	pkt/s	before	after
4800	300	3/6	6/6
8160	510	0/6	5/6
12288	780	0/6	2/6

Art-Net improves the same way (4800 lights: 3/6 to 6/6). The fix is on the shared staging path, so it is not protocol-specific. Per light Art-Net actually suffers sooner, since it carries 170 lights/packet against DDP's 480.

What we found about the limit itself

The ceiling is throughput, not packet rate. Art-Net survived 510 pkt/s where DDP failed at 300, but both broke around the same KB/s; at matched throughput, 2.6x the packet rate gave the same result. So this is bytes moved per second, not queue slots.

Transport dominates. On our bench, using the same test:

board	link	degrades at
ESP32-S3	WiFi	~440 KB/s
ESP32-S31	WiFi	>2600 KB/s
ESP32-P4	Ethernet	~10500 KB/s

Our P4 over Ethernet ran 8192 lights at 60 fps cleanly, and did not falter until far past any realistic sender. That is the one result that does not match your report, which is why the questions below matter.

Two caveats on that table so it is not read as a chip ranking: each board differs in link and in silicon, so transport and chip are not separated, and the boards carried little or no LED-driver load, a real installation has less headroom than these numbers suggest.

Will this be enough for you?

Honestly: maybe not, and it depends on your numbers. Frame rate trades directly against light count, on the S3 the usable point is roughly 4800 lights at 60 fps or ~8000 at 30 fps. Above that it still degrades. We raised the ceiling; we did not remove it.

The structural fix is to receive in a callback from the network task the instant a packet arrives, instead of polling once per render frame, which is what WLED does via AsyncUDP, and a large part of why it stays smooth on the same stream. That is a bigger change and it is not done yet.

It would help to know: your light count and sender frame rate, and whether the P4 is on Ethernet or WiFi (the WiFi P4 goes through a co-processor link that has its own separate performance problem on our side). That last one most likely explains why we could not reproduce your P4 case.

The fix will land on main / latest in the coming days. We would appreciate hearing whether it visibly improves things at your scale.

[ewowi]

merged, so you can test flashing "latest"

[ewowi]

Follow-up with numbers from a 1 Gbit Ethernet receiver, and a correction to how the earlier table should be read.

First, what "6/6" meant

That was our shorthand and we should have spelled it out: while the DDP sender runs, we fire 6 HTTP requests at the device (0.3 s apart) and count how many answer. 6/6 means the device stayed responsive under that load, 0/6 means it stopped answering entirely. It is a proxy for "is the device keeping up", not a measure of what you actually see: you reported visual stutter, and we measured HTTP because it is objective and scriptable. The two track each other, but they are not the same thing.

The gigabit Ethernet result

Same test, same sender, identical layout and light count to the earlier run (192x64 = 12,288 lights), so the wire load matches row for row. Receiver is an ESP32-S31 on a negotiated 1000 Mbit link:

lights	pkt/s	KB/s	ESP32-S3 / WiFi (before)	ESP32-S3 / WiFi (after)	ESP32-S31 / 1 Gbit eth
4,800	300	437	3/6	6/6	6/6
8,160	510	743	0/6	5/6	6/6
12,288	780	1,136	0/6	2/6	6/6

Pushing well past any real sender, at 12,288 lights (10 probes per point):

fps	pkt/s	KB/s	HTTP
idle	0	0	10/10
100	2,600	3,788	9/10
150	3,900	5,682	9/10
200	5,200	7,576	8/10
300	7,800	11,364	8/10
400	10,400	15,153	6/10

So it degrades gently rather than falling over, and only far beyond what a sender produces: 30 fps at 12,288 lights is 1,136 KB/s, and the first measurable dip is around 3,800 KB/s.

Does the gigabit link actually pay off?

Less than you might expect, because the wire was never the constraint. Against our ESP32-P4 on 100 Mbit RMII, tested the same way:

	P4, 100 Mbit	S31, 1000 Mbit
clean at	3,933 KB/s (8,192 lights)	3,788 KB/s (12,288 lights)
first dip	~10,500 KB/s	~7,600 KB/s

Both run far past any real sender, and the 10x link gives no 10x here. 100 Mbit is 12.5 MB/s of bandwidth and 12,288 lights at 30 fps is 1.1 MB/s, under a tenth of it, so what runs out first is CPU and network stack rather than the PHY. If your DDP source is at a few thousand lights and 30-60 fps, a gigabit receiver is not what will fix stutter.

Two variables changed at once here, chip and link, so this does not isolate how much of the gain is Ethernet and how much is the S31. It is the configuration you described using, which is why it is worth publishing, but we would not claim "Ethernet is 17x WiFi" from it.

A separate bug, found while testing this

The bench board would not pass Ethernet traffic at all, and the cause was a GPIO collision that nothing reported: an LED driver's clock pin defaulted to GPIO 10, which on that chip is txd2 on the Ethernet bus. Frames left the MAC counted-as-sent and arrived corrupt, while the link reported 1000 Mbit with zero drops. Two fixes went in: a bus backend no longer parks unused lanes on a real pin, and the Ethernet data pads are now published so the pin map shows them and flags anything else claiming one.

That is unlikely to be your issue (it needs an LED driver and Ethernet on the same board), but if you run both, the pin map on the System card is now worth a look.

[ewowi]

The fix is on main now, so it is in the latest pre-release build rather than something to wait for.

What landed:

• The receive effect stops re-copying a frame nothing changed. It was copying its staging buffer into the layer every tick whether or not a packet had arrived. At 12,288 lights that was 3.5 ms per tick spent writing identical bytes; now ~240 us. No frames are dropped and hold-last-frame is unchanged, which is pinned by a test.
• A follow-up fix to that fix. The first version skipped the copy whenever no packet arrived, which was wrong when something ELSE had altered the buffer: the layer runs a sibling effect's fade before the effect pass, so a held frame faded to black instead of holding. The receiver now re-lays the frame whenever the layer's buffer actually changed, which costs one integer compare.

If you flash the latest and it is still stuttery at your light count, the numbers to send back are your light count, sender frame rate, and whether the P4 is on Ethernet or WiFi. The last one matters most: the P4's WiFi goes through an on-board co-processor, and we have now measured that path costing roughly 4x per HTTP request and 2x throughput against the same board on Ethernet, even when the traffic is not on the radio at all.

We would rather hear it did not fully fix it than not hear.