Repository navigation
Credentials directory reader and AMBER_ overrides for custom sections (M-10a) - #1423
Draft
crimson-knight wants to merge 3 commits into
Draft
crimson-knight wants to merge 3 commits into
crimson-knight wants to merge 3 commits into
Conversation
…ions
Milestone M-10a. Amber::Configuration::CredentialsDirectory reads a
systemd-style $CREDENTIALS_DIRECTORY (one file per credential, file name
as key, contents as value) and builds a typed YAML::Serializable section
from it. Amber::Configuration.register_credentials registers that section
and raises CredentialsRegisteredAfterSettingsLoad when Amber.settings is
already loaded, so a misordered require fails at boot.
The reader takes the directory as an argument and never reads ENV. It
rejects path traversal, symlinks that leave the directory, non-regular
files, unreadable files and files over a size cap (64 KiB by default),
each with a named CredentialsError subclass whose message names the
credential but never its value. Values are bound as double-quoted
scalars so contents such as "null" stay strings.
The AMBER_{SECTION}_{KEY} override pass now reaches registered custom
sections, including nested mappings and nil-valued fields, using the
YAML keys captured from the registered type. The loader reads custom
sections before the override pass and rejects a YAML section that
duplicates a credentials section.
Docs cover registration, the require order for generated apps, the
reader's rules and the precedence of the layers.
Address review findings on the credentials reader: - Open each credential with O_NOFOLLOW, O_NONBLOCK and O_CLOEXEC and check its type and size on the open descriptor, so an entry swapped for a symlink or a FIFO after it was listed is rejected instead of read or waited on. Follow a symlink only when its target is a file directly inside the directory. - Read only the files a section's type declares (read_for/read_into), add read_credential?, a 1 MiB total cap (CredentialsDirectoryTooLarge) and range checks on both caps. - Map realpath and listing access errors to the named error classes. - Replace the register_credentials macro with a method that takes a Symbol or String key and keyword caps. Make the AMBER_ override pass for custom sections follow the field type: String values stay text (an empty value stays empty), Bool and Float use the built-in parsing, nested sections that are nil or absent are created when one of their variables is set, and a section is rebuilt from the YAML or credentials it was loaded from, so fields the type does not serialize keep their values.
actions/checkout@v4 and oprypin/install-crystal@v1 were movable refs that their owners can repoint. Each is now pinned to the commit its ref points to today (checkout v4.4.0, install-crystal v1.9.4), so CI runs the same code as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Milestone M-10a, the Amber credentials reader (plan:
followup-34-mica-beryl-amber-v2-plan.md, sections 6 and 7).Amber::Configuration::CredentialsDirectory(src/amber/configuration/credentials_directory.cr) reads a systemd-style$CREDENTIALS_DIRECTORY, the layoutLoadCredential=andLoadCredentialEncrypted=produce: one file per credential, with the file name as the key and the contents as the value. It takes the directory path as an argument and reads files only. It never reads ENV and never logs.readreturns every entry,read_credential/read_credential?read one entry,read_for(Type)reads only the files a section type declares, andread_into(Type)builds the typedYAML::Serializablesection.O_NOFOLLOW | O_NONBLOCK | O_CLOEXEC, and its type and size are checked on the open descriptor. A symlink is followed only when its target is a file directly inside the directory.Amber::Exceptions::CredentialsErrorsubclass, and its message names the credential but never its value:CredentialsDirectoryNotFound,CredentialsDirectoryUnreadable,CredentialPathTraversal,CredentialNotFound,CredentialSymlinkOutsideDirectory,CredentialNotARegularFile,CredentialUnreadable,CredentialTooLarge,CredentialsDirectoryTooLarge,CredentialsDoNotFitSection,CredentialsRegisteredAfterSettingsLoad.Amber::Configuration.register_credentials(key, Type, directory, maximum_file_size:, maximum_total_size:)registers the section. IfAmber.settingshas already loaded, it raisesCredentialsRegisteredAfterSettingsLoad. The loader also raises aConfigurationErrorwhen the environment YAML defines the same section.AMBER_{SECTION}_{KEY}override pass (env_override.cr) now reaches registered custom sections, credentials sections included:Stringstays text,BoolandFloatparse the same way as in the built-in pass, and other types are read as plain YAML scalars.AppConfig#custom_config_sourcesrecords, so fields the type does not serialize keep their values.SectionField(:nodoc:) lists, at compile time, the YAML fields of a section type and what kind of value each holds.docs/guides/configuration.mdcovers registration, the require order for generated apps (require "./credentials"right after"amber"and beforeconfig/database.cr), the reader's rules and how the layers take precedence. The CHANGELOG has an Unreleased entry.ci: pin GitHub Actions to full commit SHAscommit from chore(v2): clear the 25 compiler warnings so the build is GREEN #1421, byte for byte. The local dependency-pin gate refuses to push a branch whose workflow names movable tags. Both SHAs were checked against their tags (checkout v4.4.011d5960a, install-crystal v1.9.419ef8675). The commit merges cleanly with chore(v2): clear the 25 compiler warnings so the build is GREEN #1421.shard.ymlis unchanged and no dependencies were added.How it was verified
crystal-alpha spec:2446 examples, 0 failures, 0 errors, 0 pendingcrystal-alpha build --no-codegen src/amber.cr: 0 errors, 0 warningscrystal-alpha tool format --check src spec: cleancdd58b3) on every changed file: 0 failuresspec/amber/validations/params_spec.crthat are already onv2-dev. This diff adds no warnings. With chore(v2): clear the 25 compiler warnings so the build is GREEN #1421 merged in a throwaway worktree the result is GREEN:2447 examples, 0 failures, 0 errors, 0 pending, 0 warnings.O_NONBLOCKfails the FIFO specCodex review (gpt-6-sol, xhigh) and resolutions
O_NOFOLLOWand checked through the descriptor (fstat). A symlink is followed only to a file directly inside the directory, so no component below the directory can be swapped. An entry that becomes a link raisesCredentialSymlinkOutsideDirectorythroughELOOP.O_NONBLOCKon open plus a regular-file check on the descriptor. Spec: FIFO raisesCredentialNotARegularFilewithout blocking.read_intoread and checked undeclared files, so an unrelated large file failed registrationread_forreads only the type's declared keys. Spec: an undeclared oversized file and a subdirectory are ignored.maximum_total_size(default 1 MiB, systemd's total) counting names and contents, enforced while listing and reading, withCredentialsDirectoryTooLarge.to_yamlerasedignore_serializefieldscustom_config_sources), andto_yamlis used only when there is no source. Spec covers this through the loader.SectionField), and a missing nested mapping is created when one of its variables is set. Specs cover both the created case and the left-nil case.""ornullinto nil for String fieldsBool"1" andFloat"1" behaved differently from the built-in passparse_bool,to_f?), and a value that is not a number raises aConfigurationErrorthat names the variable. Spec covered.realpathpermission errors and entries removed after listing escaped the named errorsCredentialsDirectoryUnreadable,CredentialUnreadableandCredentialNotFound.Int64::MAXcap overflowedmaximum_file_size + 1Int32::MAX, otherwiseArgumentError. Spec covered.register_credentialsmacro stringified a variable nameSymbol | String.Full Codex output is kept in the AgentC report folder (
followup-37-unblocked-work/codex-amber-credentials-reader.md).crystal-conventions rubric
YAML::Serializablesections. ENV is read only by the app initializer and the existingAMBER_*pass.not_nil!/ silencing.as(T)not_nil!. The one.as(YAML::Serializable)widens the builder's return type and is not a narrowing cast.list_of_collections, question predicates (credential_section?,plain_file_name?),value_kindrather thankindCredentialsErrorbasecase … inon Grant unionsMINOR items: public methods have type restrictions and return types, and internal API is marked
:nodoc:.Follow-ups outside this PR
config/application.crmust requireconfig/credentials.crright after"amber"and beforeconfig/database.cr. That change belongs to the amber_cli repo.agentc-placeholder:marker that a local secret-scan gate requires. Maintainers can decide whether to keep it.