Repository navigation
Should ateom share network access with the actor sandbox? #240
Description
Follow-up for discussion from #122
AFAICT: We're not actually using the otel trace export ... we export to localhost, so it's effectively a no-op currently. ateom doesn't actually use the network meaningfully.
While the traces would be nice to have, we could have atelet relay these (e.g. we could use another unix socket for ateom traces => atelet), which is in-line with how we're trending for scheduling (aggregate at the atelet, avoid every worker talking directly to ate-apiserver) but repeated for tracing and the trace collector.
So far there doesn't seem to be any other use-case for inbound connections to the ateom.
I'm not sure there's a strong case for ateom to intrude on the worker's network at all, I could see potentially streaming snapshots but I think we're moving away from always pushing these over the network anyhow (#119, #227) so we'll probably have some local cache in atelet that we need to pull through anyhow.
It would be nice to just forward all traffic on all protocols from the pod IP to the interior sandbox and avoid users having to configure explicit actor ports, and more importantly avoid conflicts with any ports actors might desire.
I feel like this would've made more sense if we didn't think we'd be writing snapshots to local disk commonly (to avoid too much traffic on the nic) and if we thought we were going to eliminate atelet (#128 , currently I disagree and I think atelet will be critical for aggregating both scheduling and images / snapshots / ...), but even then debugging it may be ... confusing.
cc Tim Hockin (@thockin) Bowei Du (@bowei) Eitan Yarmush (@EItanya) Taahir Ahmed (@ahmedtd)
Activity
bowei commentedon Jun 15, 2026
EItanya commentedon Jun 15, 2026
I think to properly settle this question we need to come up with a long-term design for Ingress, as you mentioned above, and egress, like #126.
In terms of ingress I can imagine some sort of connection tracking being very useful for quiescence, but that tracking can be done in a node proxy, or a "sidecar" type of thing.
In my mind this all ties into the egress discussion in that we need to come up with an overall networking story which covers all of our bases.
BenTheElder commentedon Jun 15, 2026
I'm not convinced this is a blocker to deciding if ateom needs to use the network directly. Whether it does or not we can inject a policy proxy on the network reserved for the sandbox/actor. There should only be minor differences if the proxy needs to account for not intercepting ateom traffic ...?
Full Transparency: AIUI network policy controls are currently something security folks (Michael Taufen (@mtaufen) Taahir Ahmed (@ahmedtd)) have flagged as a GA requirement but not alpha/beta because we're convinced it's viable to extend later / don't expect major blockers.
My take:
BenTheElder commentedon Jul 29, 2026
I think we're at a place where we can conclude that no, ateom should not share the network, and this shouldn't be a blocker for ingress/egress?
Bowei Du (@bowei) I think we should document this property and ship it. So far in other discussions this seems generally agreed without any strong use cases to avoid it.
BenTheElder commentedon Oct 24, 2026
For the record: #1689
It is now safe for ateworker to use the network, however we have not chosen to do so yet, as we don't have a strong enough use case versus continuing to funnel up through atelet, which still has some attractive properties.