Add `ActorDirs` message to carry per-actor directories and and split the path packages by owner by HavenXia · Pull Request #1730 · agent-substrate/substrate

Haven Xia (HavenXia) · 2026-09-18T00:21:06Z

Today atelet and ateom derive the per-actor directories (oci bundles, runsc state, pid files, checkpoint and restore state, durable-dir, system-info and volume roots) from the actor UID through the same package, internal/ateompath.

This PR:

Add an ActorDirs message to RunWorkloadRequest, RestoreWorkloadRequest, CheckpointWorkloadRequest and TerminateWorkloadRequest, and have atelet fill it with the directories it prepared. The ateoms do not read it yet.
Split internal/ateompath into:
- cmd/atelet/internal/ateletpath: atelet's pathes, the per-actor directories (and ActorDirs, built from them) plus the directories only atelet uses.
- internal/nodepath: shared pathes, the base dir both mount, the ateom socket, the OTLP
  sockets, and the netns name.
- internal/ateompath: what the ateoms still derive from the actor UID, each function marked with the ActorDirs field it duplicates. atelet no longer imports it.

This is the first part of #1604 to decouple atelet and ateom shared pathes. Behavior does not change: both sides still compute the same paths, atelet from ateletpath and the ateoms from ateompath.

Followup PRs will remove ateompath by making ateom-gvisor and ateom-microvm read the ActorDirs message from RPC.

Benjamin Elder (BenTheElder)

so for dirs that atelet feeds to ateom, let's split those off to a package internal to atelet so we know ateoms are not assuming it? (and vice versa, atelet should stop using ateompath)

there should probably be a third package for the socket paths they share

Haven Xia (HavenXia) · 2026-09-21T18:31:24Z

so for dirs that atelet feeds to ateom, let's split those off to a package internal to atelet so we know ateoms are not assuming it? (and vice versa, atelet should stop using ateompath)

there should probably be a third package for the socket paths they share

I agree on the shape for 3 packages: atelet/internal/ for this PR's pathes, a shared path for shared pathes and an ateom specific path for ateoms.

But this PR add only the proto and RPC, the follow up PRs will do the refactors, do you want me to have them together in this PR?

Benjamin Elder (BenTheElder) · 2026-09-21T20:43:32Z

But this PR add only the proto and RPC, the follow up PRs will do the refactors, do you want me to have them together in this PR?

I think we want to reason about what these paths are and how they should be split more easily. If you don't have time a follow-up is OK, but to me that part is more important than cramming the paths into the RPCs. Otherwise you can't tell if the RPC change adequately captured the path knowledge.

Haven Xia (HavenXia) · 2026-09-22T01:03:10Z

But this PR add only the proto and RPC, the follow up PRs will do the refactors, do you want me to have them together in this PR?

I think we want to reason about what these paths are and how they should be split more easily. If you don't have time a follow-up is OK, but to me that part is more important than cramming the paths into the RPCs. Otherwise you can't tell if the RPC change adequately captured the path knowledge.

Added the refactor part into this PR, the removing of ateom itself and using the ActorDirs message from RPC is put into next PR

Haven Xia (HavenXia) · 2026-09-23T16:46:52Z

rebased over #1689, Benjamin Elder (@BenTheElder) PTAL

Benjamin Elder (BenTheElder) · 2026-09-24T00:55:23Z

@@ -0,0 +1,77 @@
+// Copyright 2026 Google LLC


Aside: we will eventually export this package when atelet/ateom settle down.

Benjamin Elder (BenTheElder) · 2026-09-24T00:57:38Z

+// to request credentials for the worker's current actor assignment.
+var AteomSupportSocket = filepath.Join(BasePath, "ateom-support.sock")
+
+// AteletOTLPSocketPath is the node-scoped unix socket atelet serves the OTLP


Note to self: we will drop the relay and just let ateom push directly following #1689 🎉

Benjamin Elder (BenTheElder) · 2026-09-24T01:03:49Z

 	tagsOwnerKind   = "tags"
 )

+// DurableDirTarFile is the snapshot file holding the tar of an actor's


This is supposed to be opaque to the snapshot tooling it should not know about the specific files.

moved to a separete snapshotfiles.go next to it

Benjamin Elder (BenTheElder) · 2026-09-24T01:06:32Z

-	return filepath.Join(LocalCheckpointsDir(actorUID), snapshotName)
-}
-
-// DurableDirTarFile is the snapshot file holding the tar of an


OK so maybe it's not for this PR, but please file a TODO, this is a really bad API, atlet should just ask for a data snapshot if that's what it wants and we should not take a full snapshot to begin with, nor know about specific snapshot filenames.

Then atelet can upload whatever files happen to be produced by the snapshot, regardless of type.

Dmitry Berkovich (@dberkov) something we should roll up with any updated snapshot lifeycle.

This should probably be a tracking issue, beyond a code todo. This is a bug that will limit our options in the future.

FYI Michelle Au (@msau42)

Benjamin Elder (BenTheElder) · 2026-09-24T02:12:01Z

+	// Holds one OCI bundle per container, named after the container.
+	OciBundleDir string `protobuf:"bytes,2,opt,name=oci_bundle_dir,json=ociBundleDir,proto3" json:"oci_bundle_dir,omitempty"`
+	// runsc --root for the actor's sandbox. gVisor only.
+	RunscStateDir string `protobuf:"bytes,3,opt,name=runsc_state_dir,json=runscStateDir,proto3" json:"runsc_state_dir,omitempty"`


TODO: can we eliminate ateom knowing about gvisor vs ...? it mostly doesn't.

for similar APIs we got rid of atelet knowing about this vs ateom knowing it. need to trace this one to see why we have to pass it between them vs deriving it in an opaque way without atelet carrying this knowledge at all.

okay, they can be removed from ActorDir as they are not directly write by atelet but only read by ateom-gvisor.

Benjamin Elder (BenTheElder) · 2026-09-24T02:12:21Z

+	SystemInfoVolumeRootsDir string `protobuf:"bytes,8,opt,name=system_info_volume_roots_dir,json=systemInfoVolumeRootsDir,proto3" json:"system_info_volume_roots_dir,omitempty"`
+	// Holds one mount point per CSI volume.
+	VolumesDir    string `protobuf:"bytes,9,opt,name=volumes_dir,json=volumesDir,proto3" json:"volumes_dir,omitempty"`
+	unknownFields protoimpl.UnknownFields


These two should be kept, atelet writes into them (it mounts the CSI volumes and generates the system-info files), and ateom-microvm reads them to for the guest paths.

Benjamin Elder (BenTheElder) · 2026-09-25T00:09:00Z

+func RunSCStateDir(actorUID string) string {
+	return filepath.Join(
+		ActorPath(actorUID),
+		"runsc-state",
+	)
+}


🤖 should-fix 🟡 – Dropping runsc_state_dir and pid_file_dir from ActorDirs removed them from the interface, but atelet still owns them. resetActorDirs (cmd/atelet/main.go:1864-1878) still wipes and creates runsc-state and pidfiles for every actor, micro-VM ones included, through this function and PIDFileDir below.

ateom-gvisor never creates either directory; it relies on atelet having made them. So the gVisor knowledge hasn't left atelet yet.

To finish the move, have ateom-gvisor create both under its root dir before it first calls runsc, the way it already does for the checkpoint dir. Then this function, PIDFileDir and that block in resetActorDirs can go.

Human: Yes that's the next PR I think.

yes, in next pr. Don't want to cause any skew between two PRs

atelet and both ateoms derive the per-actor directories (bundles, runsc state, pid files, checkpoint and restore state, durable-dir, system-info and CSI volume roots) from the actor UID through the same package, internal/ateompath. Add an ActorDirs message to RunWorkloadRequest, RestoreWorkloadRequest, CheckpointWorkloadRequest and TerminateWorkloadRequest, and have atelet fill it with the directories it prepared.

- cmd/atelet/internal/ateletpath: atelet's on-node pathes, the per-actor directories it prepares (and ActorDirs, built from them) plus the node-level directories only atelet uses. - internal/nodepath: what both sides need before the first RPC, the host directory they mount and the sockets and netns they find each other by. atecontroller renders the worker pod's hostPath from it. - internal/ateompath: only what the ateoms still uses it self.

atelet never reads the runsc state directory or the pid file directory. Both belong to ateom-gvisor, which can keep them under root_dir the way it keeps the sandbox resolv.conf, so ActorDirs no longer carries runsc_state_dir or pid_file_dir.

after agent-substrate#1843 StaticFilesDir is no longer atelet-only.

…trate#1865) This PR let ateom-gvisor reads the per-actor directories from the `ActorDirs` (added in agent-substrate#1730) in the request instead of deriving them from the actor UID. The paths `ateom-gvisor` still builds itself are put into gvisor specific `cmd/ateom-gvisor/actorpath.go`. Resetting `runsc-state` and `pidfiles` moves from atelet's `resetActorDirs` to ateom-gvisor's `main.go` to conduct at beginning of `RunWorkload` and `RestoreWorkload`, which empties them at the start of Run and Restore. Atelet doesn't actually read them, so agent-substrate#1730 drops them from `ActorDirs`. Added a new `ActorDirs` validation to reject a request with `InvalidArgument` when `ActorDirs` is missing or any directory in it is empty, relative or unclean. The gVisor paths are removed from `internal/ateompath`; the rest goes with ateom-microvm in the next PR. Part of agent-substrate#1604.

… ateompath package (agent-substrate#1872) This PR let ateom-microvm reads the per-actor directories from the `ActorDirs` (added in agent-substrate#1730) in the request instead of deriving them from the actor UID. The paths ateom-microvm still builds itself are put into microvm specific`cmd/ateom-microvm/actorpath.go`. The `ateompath` package as nothing uses it anymore. Fix agent-substrate#1604

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch from 3295471 to 0adaa55 Compare September 18, 2026 00:24

Haven Xia (HavenXia) requested a review from Benjamin Elder (BenTheElder) September 18, 2026 00:24

Haven Xia (HavenXia) added area/api User-facing API changes area/api-machinery labels Sep 18, 2026

Benjamin Elder (BenTheElder) reviewed Sep 18, 2026

View reviewed changes

Haven Xia (HavenXia) assigned Benjamin Elder (BenTheElder) Sep 21, 2026

Haven Xia (HavenXia) requested a review from Benjamin Elder (BenTheElder) September 21, 2026 18:31

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch from 0adaa55 to 8f64737 Compare September 21, 2026 18:37

Julian Gutierrez Oschmann (juli4n) reviewed Sep 21, 2026

View reviewed changes

Comment thread internal/proto/ateompb/ateom.proto Resolved

Haven Xia (HavenXia) changed the title ~~Add ActorDirs message to carry per-actor directories~~ Add ActorDirs message to carry per-actor directories and and split the path packages by owner Sep 22, 2026

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch 2 times, most recently from bc33f8f to 9529d68 Compare September 22, 2026 01:02

Haven Xia (HavenXia) requested a review from Julian Gutierrez Oschmann (juli4n) September 22, 2026 01:02

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch from 9529d68 to 65ee20b Compare September 23, 2026 16:46

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch 3 times, most recently from 354b0eb to 865c6a7 Compare September 24, 2026 00:06

Haven Xia (HavenXia) mentioned this pull request Sep 24, 2026

[WIP][Stack on #1730] Use ActorDirs in ateom-gvisor/ateom-microVM and remove ateompath #1842

Closed

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch from 865c6a7 to b4eec9b Compare September 24, 2026 00:41

Benjamin Elder (BenTheElder) reviewed Sep 24, 2026

View reviewed changes

Comment thread kubeconfig Resolved

Benjamin Elder (BenTheElder) reviewed Sep 24, 2026

View reviewed changes

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch from b4eec9b to 438c333 Compare September 24, 2026 01:07

Haven Xia (HavenXia) requested a review from Benjamin Elder (BenTheElder) September 24, 2026 01:39

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch from b151a8b to 79364bd Compare September 24, 2026 01:58

Benjamin Elder (BenTheElder) reviewed Sep 24, 2026

View reviewed changes

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch 2 times, most recently from ccd8622 to 200609c Compare September 24, 2026 04:02

Haven Xia (HavenXia) requested a review from Benjamin Elder (BenTheElder) September 24, 2026 04:02

Haven Xia (HavenXia) mentioned this pull request Sep 24, 2026

ateom-gvisor: take the actor directories from the request #1865

Merged

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch 2 times, most recently from f56f1ed to c3be398 Compare September 25, 2026 00:12

Haven Xia (HavenXia) mentioned this pull request Sep 25, 2026

ateom-microvm: take the actor directories from the request and remove ateompath package #1872

Merged

Benjamin Elder (BenTheElder) approved these changes Sep 25, 2026

View reviewed changes

Benjamin Elder (BenTheElder) previously approved these changes Sep 25, 2026

View reviewed changes

Benjamin Elder (BenTheElder) added this pull request to the merge queue Sep 25, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 25, 2026

Haven Xia (HavenXia) added 5 commits September 25, 2026 13:30

resolve comments

5df80a4

Move StaticFilesDir to nodepath

08a7a25

after agent-substrate#1843 StaticFilesDir is no longer atelet-only.

Haven Xia (HavenXia) dismissed Benjamin Elder (BenTheElder)’s stale review via 08a7a25 September 25, 2026 20:35

Haven Xia (HavenXia) force-pushed the ateom-actor-dirs branch from c3be398 to 08a7a25 Compare September 25, 2026 20:35

Benjamin Elder (BenTheElder) approved these changes Sep 25, 2026

View reviewed changes

Benjamin Elder (BenTheElder) enabled auto-merge September 25, 2026 20:55

Benjamin Elder (BenTheElder) added this pull request to the merge queue Sep 25, 2026

Merged via the queue into agent-substrate:main with commit a0b680d Sep 25, 2026
11 checks passed

Haven Xia (HavenXia) deleted the ateom-actor-dirs branch October 28, 2026 16:26

Haven Xia (HavenXia) restored the ateom-actor-dirs branch October 28, 2026 16:27

Conversation

Haven Xia (HavenXia) commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Benjamin Elder (BenTheElder) left a comment

Choose a reason for hiding this comment

Uh oh!

Haven Xia (HavenXia) commented Sep 21, 2026

Uh oh!

Uh oh!

Benjamin Elder (BenTheElder) commented Sep 21, 2026

Uh oh!

Haven Xia (HavenXia) commented Sep 22, 2026

Uh oh!

Haven Xia (HavenXia) commented Sep 23, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Haven Xia (HavenXia) Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Haven Xia (HavenXia) Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Haven Xia (HavenXia) commented Sep 18, 2026 •

edited

Loading

Haven Xia (HavenXia) Sep 24, 2026 •

edited

Loading

Haven Xia (HavenXia) Sep 24, 2026 •

edited

Loading