Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions reviewer/GIT-DESCRIPTOR-BOUNDARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Git worker descriptor observation

The private descriptor observer supplements process and mount checks before the
private [factory](GIT-FACTORY.md) grants work. It returns no source or reusable admission
capability. The caller retains the original worker pidfd and rechecks its
identity before and after observation.

The observer enumerates bounded kernel `fdinfo`, reads the original descriptors'
current flags, and duplicates actual objects through that original pidfd. A
duplicate always has CLOEXEC itself, so its descriptor flags cannot establish
the original CLOEXEC bit. Reading kernel `fdinfo` does. See Linux
[fdinfo](https://man7.org/linux/man-pages/man5/proc_pid_fdinfo.5.html) and
[pidfd_getfd](https://man7.org/linux/man-pages/man2/pidfd_getfd.2.html).

Inspection never reads object contents or changes their offsets, status flags,
socket options or connection state. It checks the object again through the
original pidfd and requires stable original flags, identity and descriptor
numbers. Any permission error, unknown descriptor or observed change refuses.
This is a readiness check on the measured inert worker before candidate code
runs, not a general guarantee against an adversary changing a running process.

The fixed worker has null standard streams. All private descriptors require
CLOEXEC. The exact export write pipe must match the factory's retained read
endpoint. Exactly one control connection must be an unnamed AF_UNIX SEQPACKET
client connected to the original host pathname of the factory's actual
accepted control connection, with the current supervisor's kernel peer
credentials. The namespace-local `/broker/worker.sock` alias does not change the
kernel's stored server pathname. The accepted connection's original peer pidfd
must match the retained worker. Exactly one pidfd must match the retained original supervisor.
The inventory must contain exactly eight descriptors, including exactly one
CLOEXEC eventpoll and one CLOEXEC eventfd for the pinned Go runtime. Extra
poller handles are not admitted merely because they share those anonymous types.
Extra sockets, directories, regular files, pipes, pidfds and other anonymous
objects refuse, even when they have CLOEXEC. Socket domain checks never exercise
inherited Internet sockets.

## Validation

Static policy fixtures cover role replacement, missing or duplicate descriptors,
wrong access modes, missing CLOEXEC, extra host files/directories/sockets,
Internet domains and unknown anonymous objects. These are parser/policy tests.
The ordinary inert child test waits for its kernel sleep, then inspects actual
null descriptors without leaking handles. Waiting for sleep avoids inspecting
the dynamic loader's temporary library descriptors before inert readiness.

An explicit private-namespace fixture runs an ordinary measured Go executable,
authenticates the original supervisor and disables dumpability. It supplies a
real export pipe and tests the fixed roles plus actual extra host file,
directory, Internet socket, inheritable descriptor and extra eventfd/epoll cases. It uses trusted
empty fixture storage, never candidate input or a production storage fallback.

```sh
SFL_GIT_DESCRIPTOR_KERNEL=1 SFL_GIT_BOOTSTRAP_NAMESPACE=1 \
go -C reviewer test -race -count=1 -v -run '^TestGitDescriptorActualPrivateWorker$' .
```

Mini denies traversal of the nondumpable worker's `/proc/PID/fd` directory.
The implementation enumerates `fdinfo` and uses original-pidfd duplication;
it does not relax dumpability, change host policy or substitute pathname opens
for retained kernel object identities. The explicit fixture's actual result
must accompany any qualification claim. The ordinary suite skips this opt-in.

Inspection also has an explicit nonancestor fixture. An inert launcher exits
before observation, and the test checks the worker's actual ancestry through
PID 1. The worker verifies dumpability is zero before sending readiness. The
test checks the target user namespace's owner and its parent chain back to the
inspector, then exercises actual root, executable, mappings, mount and original
pidfd descriptor observation. No service, host capability or ptrace-policy
change is made.

Linux grants a parent user namespace's matching owner capabilities in its child
and descendant namespaces. Ptrace's dumpability check permits `CAP_SYS_PTRACE`
in the target's user namespace. These rules differ from host capabilities and
do not depend on process ancestry. See
[user namespaces](https://man7.org/linux/man-pages/man7/user_namespaces.7.html)
and [ptrace access checks](https://man7.org/linux/man-pages/man2/ptrace.2.html).
The retained pidfd identifies the process; it does not override access checks.
Other host security policies can still refuse, and refusal never qualifies.

```sh
SFL_GIT_DESCRIPTOR_KERNEL=1 SFL_GIT_BOOTSTRAP_NAMESPACE=1 \
go -C reviewer test -race -count=1 -v \
-run '^TestGitDescriptorActualNonancestorNondumpableWorker$' .
```

This does not qualify sealed worker execution, the required whole-unit noswap
tmpfs, installed-filter provenance, unit/cgroup ownership, authenticated export
publication or controller recovery. Those remain separate factory requirements
in #176. No provider credentials, model inference or live activation occur here.
27 changes: 25 additions & 2 deletions reviewer/GIT-EXPORT-ARCHIVE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,9 @@ An existing canonical private service directory is anchored with `os.Root` and
owned for the archive's lifetime by a process flock. A second process cannot
open it concurrently. Root/lock identity, ownership and private permissions are
checked; unknown root entries or malformed record directories require recovery.
At most 128 spent attempt directories are retained. The factory must implement
qualified retention/cleanup; this archive does not discard evidence to free space.
At most 128 source attempt directories are retained. Only the concrete factory
can retire unreferenced terminal attempts after durable job retention and
consumer-lifetime checks. The archive never discards evidence solely for space.

Each `export-<32-hex-attempt>` directory contains only fixed `archive.json` and
`payload.bin` names. Attempt names never come from a worker path. The canonical
Expand Down Expand Up @@ -93,3 +94,25 @@ qualification, physical-stop/recovery authority, generation leases, controller
composition and live activation remain independent work. See
[GIT-MATERIALIZATION.md](GIT-MATERIALIZATION.md), [GIT-UNIT.md](GIT-UNIT.md) and
[GIT-RUNTIME.md](GIT-RUNTIME.md).

## Factory recovery integration

The private [factory](GIT-FACTORY.md) now binds staged exports to the exact
unit journal and bootstrap/worker grants before restored source can escape.
An incomplete receiving intake may become terminal `discarded` only after
the concrete owner independently proves physical whole-unit stop. That receipt
is fsynced before removing the bounded owned payload; a restart can complete
cleanup. The nonce stays spent and discarded intake cannot reopen as source.
A different nonce may stage after that recovery. Complete staged source is
retained while a pending capture, active consumer or durable job needs it.
The factory can retire it after those references expire, using a fsynced intent
and atomic whole-directory moves into archive-owned retirement storage. The
archive recognizes at most 128 intents and their two fixed retirement directories
per intent, with exact original directory identities. Partial deletion stays
recoverable without parsing a half-deleted source record. Unexpected files or
replacements refuse cleanup. See [factory retention](GIT-FACTORY.md).
This is no public force-delete or arbitrary retry interface.

Close disposes both lock and root descriptors once intake has joined, records
terminal state even if disposal reports an error, and reports that initial
error. A timeout acquiring the operation token still retains actual ownership.
192 changes: 192 additions & 0 deletions reviewer/GIT-FACTORY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# Git acquisition factory

The private source factory composes the existing acquisition unit, pinned
credential generation, measured runtime, authenticated bootstrap, worker and
plain export archive. The polling controller takes it through `PollOptions.Source`
so capture, restore, consumer release and retention share one owner. Work roots
must have different opened directory identities from both journal roots; aliases
refuse before transferring ownership. Raw
Capture/Restore callbacks remain for fakes; they cannot be mixed with Source.
The trusted service must construct and retain the factory;
this change does not install a service, import credentials or activate polling.

Construction takes exclusive lifecycle ownership of a newly opened unit owner,
export archive and prepared runtime. It validates the fixed manifest/helper and
runtime pins and a shared filesystem for the unit/archive journals, then
performs startup recovery before requesting any credential
generation. The composing service must call this before credential Refresh and
retain the factory until controller work has joined. Before local handles are retained, a nil factory with an error leaves inputs
owned by its caller. After ownership transfers, a failed startup returns a
complete nonnil factory with the error. Retain that handle for explicit Recover
or Close; Capture and Restore remain fenced until recovery succeeds. Unknown
units keep their runtime and owner lock. Credentials remain shared
with the publisher; the source factory never exposes a raw token or refresh
capability to a worker or broker.

## Capture and grant order

Credential readiness is checked before allocating any durable slot or work
directory. Each admitted capture spends a random 128-bit attempt nonce. Its binding fixes the
release hash, canonical runtime-manifest hash and immutable PR head/base/merge
base. A fresh private attempt directory contains separate bootstrap and broker
sockets. No failed or uncertain attempt automatically starts again. Unix pathname bounds
are checked against each actual bootstrap/worker/broker socket path, including
the fixed nonce directory. The bootstrap endpoint must fit the unit owner's
100-byte limit; broker endpoints must fit the kernel's 107-byte limit.

The factory seals the fixed non-executable manifest, duplicates measured runtime
and helper handles, and retains the original sealed worker image and supervisor
pidfd from the unit owner. The bootstrap receives only the existing eight
descriptor roles. The broker has the typed reader for the one pinned credential
generation; it has no publisher, refresh, raw token or arbitrary HTTP interface.

After the broker is listening, the factory starts the fixed unit once and
authenticates the bootstrap's canonical hello against its actual original peer
pidfd. The unit owner verifies the root, manager, invocation, exact cgroup and
fixed resource properties and fsyncs the bootstrap grant before sending it.
The parent then closes its export write endpoint so it cannot manufacture a
missing EOF.

Worker readiness is a canonical packet bound to that same manifest and attempt.
Before the worker grant, the factory checks:

- The original live root and unchanged manager, invocation, unit object path,
resource policy and independently captured cgroup.
- A retained ancestor chain from the original worker through only the measured
namespace helper to the original unit root. Every retained ancestor maps to
the exact unit and invocation; parent edges and executable inodes are rechecked.
- The measured worker/runtime, six private namespaces, capability/map/network
policy and original peer identity in the existing process boundary.
- The actual fixed mount/device/private-proc layout and workspace limits in
[mount observation](GIT-MOUNT-BOUNDARY.md).
- Actual inherited objects, current CLOEXEC bits, exact export pipe, original
supervisor pidfd and the accepted control connection in
[descriptor observation](GIT-DESCRIPTOR-BOUNDARY.md).

Installed-filter provenance comes from the measured fixed bootstrap and helper,
the sealed fixed filter in the authenticated bootstrap grant, and the fixed
worker launch path. `Seccomp=2` alone is not the filter's identity or a sandbox
qualification. The fixed code supplies no caller-selectable filter or argv.

Each broker request separately checks measured Git/helper ancestry and fresh
unit ownership before any GitHub dispatch. Admission is a private concrete
composition. No exported option can replace these checks with a model claim or
caller-provided boolean.

## Stop and publication

The plain export is staged and fsynced while the acquisition still owns the
credential generation. Complete framing, valid hashes and EOF do not publish it.
The immutable stop capability first revokes work and cancels the acquisition,
then joins the sole broker, all its callbacks and the acquisition method. It
next proves whole-unit physical stop with the independent unit journal. Only
the credential owner can accept that proof and release the generation lease.
The factory closes retained run assets before returning a materialization.

An early broker setup refusal with a joined zero-dispatch result has no live
broker to retain. A callback or serve goroutine that has not returned keeps
ownership. A lost start reply, populated group, failed physical stop or unknown
outcome keeps the generation, runtime and service locks for explicit recovery.
Close can retry that exact retained generation; it never replaces it with a
different credential pair or drops an unknown unit.

Before returning SnapshotEvidence, capture reconstructs patch anchors from
verified plain blobs. It shares the same packet validation used for restoring
an admitted job, without inventing a job or running host Git. Restore matches
the exact admitted packet and snapshot, rechecks the retained attempt's original
release/runtime binding across both durable journals and reopens the bounded
archive into sealed blobs. New capture assets do not replace an older attempt's
provenance. The controller qualifies the job's admitting release before Restore,
including after an upgrade; a revoked release exposes no restored source. A
canonical protected stopped receipt remains valid after systemd collects the
unit and cgroup; an unfinished stop still needs independent live proof. There
is no network refetch, host Git parser, recapture or label-based fallback.

## Restart recovery

Startup joins the newly opened cross-process-locked unit owner with the export
archive. It completes durable retirement intents, independently stops every
remaining recorded old acquisition, then requires
exact binding equality between journals. A staged export needs both durable
bootstrap and worker grants from a captured unit. An orphan, different binding
or archive without worker admission refuses startup source recovery.

An incomplete receiving frame can be discarded only after the concrete unit
owner proves physical stop. A canonical terminal `discarded` receipt is fsynced
before removing the bounded owned payload. Recovery can repeat after a crash;
the nonce remains spent, and discarded intake can never be reopened as source.
A later fresh nonce may stage after the incomplete attempt becomes terminal.
The same retirement runs after a failed capture in the current process, once
its physical stop and credential-generation acceptance are known. Unknown or
corrupt outcomes fence Capture/Restore until explicit Recover first drains the
retained local runs; recovery does not restart them.
Staged admitted source is retained by the separate retention rules below.
All existing 128-record bounds remain. The factory retains its private working
directories as recovery artifacts; it does not recursively delete a replacement
or foreign directory.

## Consumer lifetime and retention

Capture pins its evidence until admission or fencing has finished. Restore pins
the returned job evidence until review and publication preparation have joined.
The polling controller releases both on success and failure, using a bounded
uncanceled cleanup context. Closing the factory refuses while either consumer
remains. A caller outside polling must perform the same releases.

During ordinary polling, before work and after consumer release, retention calls
SQLite's existing Prune and reads the remaining immutable jobs. Idle polls also
retire expired source without requiring another review job. Polling skips the
source retention hook while a worker owns capture, so it does not wait for the
factory operation token or turn a healthy long acquisition into tick backoff.
Before/after-work retention and later idle polls still collect eligible source. Pending captures,
active job evidence, every retained job and every unaccepted acquisition keep their source. An
expired lease alone never permits deletion. Unused sealed caches can close
while the plain archive remains available for exact restore. Retained jobs with
an outstanding worker lease keep their cache as well.

Only unreferenced, revoked, physically stopped attempts can retire. The factory
fsyncs an intent containing the exact terminal unit receipt, export receipt and
original directory/file identities and hashes. It atomically moves the complete
unit directory into archive-owned retirement storage, then the export directory,
before deleting known files. Both journal roots must share a filesystem. This
keeps an interrupted deletion out of the unit owner's normal record scan.
The intent remains until both directories are removed and synced. Recovery
resumes it before checking ordinary unit/export correspondence. Unexpected
entries, changed identities, mismatched content or cancellation retain the
intent and fence new work. No recursive deletion or external path is accepted.

Retirement removes terminal journal slots and in-memory run/binding/cache entries,
so 128 is a simultaneous retained-record limit rather than a lifetime quota.
Fresh captures still generate their own random nonces; there is no model-facing
retry or nonce selector. Evaluation consumers must retain their capture or job
references throughout all sessions that need them, including recovery.

## Validation and remaining gates

Offline tests use fake managers/cgroups and local synthetic credential transports.
Actual processes, pipes, original pidfds, flock, sealed files, archive fsync and
small trusted Git repositories are distinguished from those fake observations.
They cover stop-before-publication, unknown-start/stop retention, cancellation,
broker drain, unreturned broker work, setup refusal, exact live-unit rebinding,
interrupted intake retirement, orphan/missing-grant recovery refusal, startup
before credentials and retained packet/anchor restoration after deleting the
synthetic quarantine. The separate actual namespace fixtures measure mount and
descriptor subsets using an ordinary inert executable.
Retention tests exceed 128 stopped attempts, preserve unreleased durable leases,
active consumers and pending admission, and reopen after six modeled deletion
boundaries. A real owner process is SIGKILLed after the durable unit-directory
rename; fresh locked owners complete its interrupted retirement. Foreign file
and directory replacements remain untouched.

The complete fixed unit has not passed physical qualification on mini. Its
required systemd noswap tmpfs fails before payload execution with
`226/NAMESPACE`, and sealed inner-worker execution independently fails with
`EACCES`. No production workspace fallback, policy relaxation or fabricated
qualification was added. Successful inert namespace checks do not waive these
failures. See [launch constraints](GIT-LAUNCH-CONSTRAINTS.md).

Trusted service/release packaging and promotion, controller startup wiring,
provider public-target/account/auth/billing evidence, native-control evaluation,
rendered GitHub evidence and explicit owner authorization for live inference
and activation remain separate gates in #172, #173, #174, #176 and #177. This is
offline source implementation; it changes no App configuration or live service.
Loading