TOML settings and kci tool

[gctucker]

Now we have a clearer idea of what features the kci command line tool needs to be able to provide, and we know there are CLI frameworks that can help us implement them easily, here's a proposal for how the TOML settings would work alongside the kci tool. Other tools such as pipeline services should be able to reuse this by importing the kernelci.cli package.

Roadmap issue: kernelci/kernelci-core#2125

• TOML settings rework kernelci-core#2129

[JenySadadia]

+1 for this proposal.
I liked the idea of dropping DEFAULT section and having different namespace names.
Also, having a separate section for secrets makes sense.

[gctucker]

Thanks for the feedback.

Note: I just moved the sections from the original description to separate comments as that makes it easier to discuss them in separate threads.

[gctucker]

Default values in TOML settings

Just like the legacy kci_* tools load default values for their arguments in the settings file, we can rely on the TOML settings file for this too. For example, kci whoami --api=early-access could be simplified to kci whoami with the following entry in TOML

[kci.whoami]
api = 'early-access'

Values defined higher in the hierarchy would be inherited by sub-commands, so this would do the same but also provide a default value for all the kci sub-commands. Then command line arguments would take precedence in any case. Here's an example with a particular exception for one subcommand:

[kci]
api = 'early-access'

[kci.special.command]
api = 'something-else'

That way, we can drop the implicit 'DEFAULT' section as it will all be based on the kci root command line section. If there are other tools using the same TOML file they can have their own "namespace" too by having another root section.

[gctucker]

Secrets in TOML settings

While the legacy tools reused the feature of loading default values from the settings to store secrets, this proposal makes a clear distinction between them. One key aspect is that secrets aren't defined as command line arguments any more, so no --api-token argument on the command line. This simplifies the syntax and avoids insecure usage anyway as the command line is visible on the host system.

Instead, a [kci.secrets] section can be used in the TOML file to store such secrets. Then the subsections can be looked up with some helper code based on the config being used. For example, the --api argument is used to specify the API config (loaded from YAML with server URL, version etc.) and this config name can be reused to store the secret token in TOML:

[kci.secrets]
api.early-access.token = 'secret-token-1234abcd'
api.other.token = 'other-secret-token'
storage.staging-azure.credentials = 'some-storage-secret'
storage.local-ssh.credentials = 'some-other-storage-secret'

So when using --api=early-access, the implementation can look up a secrets.api.token which will automatically resolve as kci.secrets.api.early-access.token in the TOML settings (see Click RFC):

def whoami(config, api, secrets):
    configs = kernelci.config.load(config)
    api_config = configs['api_configs'][api]
    api = kernelci.api.get_api(api_config, secrets.api.token)
    data = api.whoami()
    click.echo(json.dumps(data, indent=2))

From a user point of view, this would work seamlessly:

$ kci whoami --api=early-access
# prints early-access user info
$ kci whoami --api=localhost
# prints local instance user info

as long as the tokens for these two API configs are present of course.

[gctucker]

Legacy deprecation

The legacy tools and settings file are meant to be deprecated at the same time as the legacy KernelCI infrastructure, so the new kci tool can just adopt TOML with a simpler implementation following the proposal above. Users who have already adopted TOML with the legacy tools will be able to adjust their settings files accordingly and without too much complications. A guide or how-to can be provided for this as part of the documentation to migrate to the new API.

[gctucker]

OK this has been reviewed and implemented. Some small things might still need to be tweaked, and the documentation updated but this discussion was to establish the design for TOML settings and that's now done.