Repository navigation
feat(gateway): support drop-in configuration fragments (gateway.d/*.toml) #5446
Description
User Story
I run the packaged OpenShell gateway and customize a few settings, such as driver options. I need those customizations to live apart from the package-seeded gateway.toml, so I don't have to reapply them after a package change or migration.
Problem Statement
The gateway reads exactly one TOML file, and TOML cannot be layered. The rpm seeds ~/.config/openshell/gateway.toml from a vendor default and migrates it on upgrade only while it is unedited. Any customization therefore means editing that file, which turns it into an operator-owned file the package no longer updates.
Impact / Why This Matters
- Operators who edit the file stop receiving updated package defaults.
- Operators who replace it lose the defaults and migrations altogether.
- Automation that needs to change a few settings, such as the tmachine test harness, carries a merge script (
tests/tmachine/ansible/roles/openshell_gateway/files/merge-gateway-toml.py) to do what the gateway cannot.
Proposed Design
- The gateway loads its base file plus
*.tomlfragments from agateway.d/directory, in lexical order. - Tables merge key by key; scalars and arrays replace.
- The merged result is validated once (version, unknown fields, secrets in the file), and errors name the fragment file.
- Precedence becomes flag > env > fragments > base file > built-in default.
openshell-gateway config preflightloads the same merged result and lists the files it merged.- Operator fragments stay separate from package defaults, so a package can update its defaults without touching operator changes.
This implements open question 1 ("Directory-based config (conf.d pattern)") of RFC 0003 (proposed in #951, implemented in #1317). The RFC defers it until three decisions are settled, which this work must resolve: whether --config and --config-dir are exclusive or composable and which takes precedence; whether a later file's array replaces or appends; and that deny_unknown_fields validation applies to the merged result rather than each fragment.
Prior art to examine before settling the semantics (their exact behavior is to be verified, not assumed): cri-o's crio.conf.d drop-in directory, containerd's config imports, systemd-style *.d override directories, and the kubelet's config directory. Compare how each handles arrays, unsetting a key, ordering, and a vendor directory plus an admin directory.
Suggested UX
# ~/.config/openshell/gateway.d/10-podman.toml
[openshell.drivers.podman]
userns = "keep-id"Acceptance Criteria
- A fragment overrides a key the base file sets, and adds a table the base lacks.
- Fragments apply in lexical order, and a later fragment wins.
- Preflight and startup report the same merged result and fail on the same errors.
- Errors name the file that caused them.
-
docs/how-it-works/gateways/configuration.mdx,deploy/rpm/CONFIGURATION.mdand the man page describe the directory, its order and its merge rules.
Open Questions
- Is the directory implicit (next to the config file) or selected by
--config-dir/OPENSHELL_GATEWAY_CONFIG_DIR? Does an explicit--configdisable it? - How does a fragment unset a key the base sets, since TOML has no null?
- Do fragments need
[openshell] version? - Should a vendor directory under
/usr/shareand an admin directory both be honored, with systemd-style override order? - How does it map to Helm ConfigMap and Secret mounts?
Alternatives Considered
- Per-key
OPENSHELL_*environment overrides: they exist for many gateway settings, but not for nested driver tables, arrays or most of the schema. - Repeated
--configflags where the later file wins: this fits a systemdExecStart, but needs the same merge and validation design and an edited or overridden unit. - containerd-style
importsin the base file: this puts the list of fragments in the file the operator is trying not to edit. - Keep merging in scripts: fragile, and it hides the effective configuration.
- Replace the file wholesale: this loses package defaults and migrations.
Activity
TaylorMutch commentedon Nov 10, 2026
My preference is for
--config-dirand--configto be mutually exclusive: allow one or the other, and reject specifying both. That makes the choice explicit:--config-dirselects directory-based configuration using the conf.d pattern, while--configpreserves the existing single-file behavior.elezar commentedon Nov 10, 2026
If the goal is to allow users to override configs by using drop-in files how would this work? Would they be expected to modify the mechanism that they use to start the gateway? Would it not make more sense to allow
--configto define the base config, and anything in--config-dirto override these settings based on a predefined ordering.elezar commentedon Nov 10, 2026
Prior art check on how other tools handle a base config plus drop-ins. CRI-O, containerd and kubelet were checked against their docs and source; the systemd row is from general knowledge and not verified.
crio.confpluscrio.conf.d/*.conf.--configand--config-dir(default/etc/crio/crio.conf.d).storage_options, which is appended and de-duplicated. There is no way to unset a key.imports = ["/etc/containerd/conf.d/*.toml"]insideconfig.toml. There is no directory flag.--configplus--config-dir. Drop-ins use the.confsuffix and are YAML/JSON withapiVersionandkind. No default directory.map[string][]string, where the whole list per key is replaced. Validation runs only on the final merged result. There is no unset.unit.d/*.confin vendor (/usr/lib) and admin (/etc) directories./etcshadows/usr/lib.What this suggests for the open questions
--configvs--config-dir. CRI-O and kubelet compose them:--configis the base and the directory layers on top. That fits the stated goal, since operators can override packaged defaults without changing how the gateway is started. Making them mutually exclusive would be the outlier.importsputs the fragment list in the file operators are trying not to edit. CRI-O and kubelet avoid this with a flag or default directory.storage_options.deny_unknown_fieldsapplies to the merged config rather than each fragment.Not checked: whether CRI-O rejects unknown keys in an individual drop-in.
TaylorMutch commentedon Nov 10, 2026
Fair enough, just was thinking about how to keep things simple without complexity, but I'm okay with allowing them to be composed if it doesn't introduce much complexity.
feat(gateway): merge drop-in config fragments over the base file
test(tmachine): apply gateway overrides as a drop-in fragment