Repository navigation
Packet size limit exceeded when forwarding back to UDP client from TCP/TLS server #430
Description
When stubby receives a request from an UDP client without an edns0 option, and it forwards it via TLS or plain TCP, if the reply is larger than 512 bytes, stubby will forward the large reply back to the client which will reject the reply.
This happens when stubby is used on openwrt routers for example, since /etc/resolv.conf does not contain "options edns0", so the packet limit is 512 bytes:
https://cgit.uclibc-ng.org/cgi/cgit/uclibc-ng.git/tree/include/arpa/nameser.h#n77
Using name compression on the reply would alleviate this issue since most TLS-DNS servers get the reply from an authoritative server which fits in 512 bytes.
I can work on adding name compression in stubby, but does the getdns API provide a way to implement this ?
Reacted by Max Mazurov and Andrei Shevchuk
Activity
foxcpp commentedon Oct 17, 2019
I don't think name compression is going to solve this completely. Some responses are just bigger than 512 bytes (see example below).
stubby/getdns should truncate packets to buffer size advertised by client or 512 bytes if EDNS is not supported by client.
Ref.: https://tools.ietf.org/html/rfc5625#section-4.4
Here is the example that caused problems in my case (non-EDNS-capable client was confused by large response):
wtoorop commentedon Feb 13, 2020
Completely agree...
We'll have to record client EDNS buffer size and send packet if it fits it, or reply truncated.
Issue #430: Record and guard UDP max payload size with servers.
wtoorop commentedon Mar 3, 2020
Closing because it's in getdns-1.6.0
epilsits commentedon Jul 15, 2020
I've observed that when truncation happens the response does not contain any answer records. Is this to be expected?
Myself and several other users have experienced problems with IoT devices (nest protect and SmartThings) after this patch. My experience was that my nest protect's were unable to connect to the Nest network and I could not add new devices to my account. I ran tcpdump on my router (rt-ax88u running merlin's asuswrt with DoT enabled) and saw the response from stubby had the TC bit set but the answer section was empty. Turning off DoT allowed the devices to connect again.
The request was for frontdoor.nest.com and the response is 792 bytes, using cloudflare's DoT servers. The nest does not seem to be EDNS capable, as the request did not have the EDNS OPT section, nor does it retry the query in TCP mode. The truncated response was something like ~32 bytes.
I'm having a hard time reconciling the RFC recommendation here, seeming to recommend not truncating the response.
Since UDP packets larger than 512 octets are now expected in normal operation, proxies SHOULD NOT truncate UDP packets that exceed that size. See Section 4.4.3 for recommendations for packet sizes exceeding the WAN MTU.Is there any middle ground here? Perhaps a set of configurations? Or is there something wrong with the patch, and there actually should be something in the truncated answer section?
vttale commentedon Jul 15, 2020
It can happen, yes, even beyond the far most common case of a delegation being provided (which only has records in the authority section and maybe additional section). If the full rrset that would go in the answer section, plus the header, exceeds 512 bytes, then a standards compliant server would remove the whole rrset and set the truncate bit (tc). Nameservers can also optionally remove everything (but the header) and set tc anyway; this is how response ratelimiting works, for example.
Hmm, I'm curious about what the answer is expected to be in the absence of the problem at hand. From my location when I ask for frontdoor.nest.com at the nest.com servers I get back a perfectly reasonable response of one CNAME; when fulfilling the chain then even including the 8 A records it still all fits in 512 just fine.
Taking getdns out of the picture, what are the Cloudflare servers saying?
RFC 5625 was concerning itself with the issue of proxies (and firewalls) that would drop DNS/UDP packets over 512 bytes while completely oblivious to EDNS and larger DNS/UDP packets. If the Nest is not asking with EDNS then a server needs to truncate at 512.
epilsits commentedon Jul 15, 2020
Here's what I get locally. This is on an asus RT-AX88U running Merlin's asuswrt with DoT enabled (dnsmasq > stubby) using cloudflare's DoT servers.
And dnsmasq cached response.
And cloudflare direct.
epilsits commentedon Jul 16, 2020
Here's a capture I had while debugging. This is the response from stubby -> dnsmasq.
vttale commentedon Jul 16, 2020
Interesting, that is indeed a similar problem as mentioned in the opening of this issue, the lack of name compression causing the response to grow much larger.
vttale commentedon Jul 16, 2020
Willem, what's the rationale for choosing to not use compression? I understand well that name compression would not address every response; of course some need tc. But why is it not being used in general?
epilsits commentedon Jul 28, 2020
Any additional thoughts here? It leaves me and other users in a tough spot where continuing to use DoT via stubby knocks a bunch of IoT devices offline. The original issue is also valid, but clearly there are a whole class of devices that also don't handle the "correct" behavior.
wtoorop commentedon Jul 29, 2020
sorry, I was on vacation. I'll have a look soonish.
wtoorop commentedon Jul 29, 2020
Okay, reading everything above I think we should add:
epilsits commentedon Jul 29, 2020
I think that would be very helpful.
Issue #430 listeners reply returned wireformat
mavack commentedon Sep 29, 2020
Have tested this update on openwrt, i found personally that many office365 domains with their CNAME records would result in our F5 big-ip client not understanding the replies when > 512 even thou perfectly valid replies. Running current getdns-dev seams to have resolved that.