Add unit tests for osism/tasks/openstack.py#2459
Conversation
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The
test_openstack.pymodule is quite large (~1600 lines) and covers many distinct concerns (cloud env, connections, baremetal, NetBox, managers); consider splitting it into multiple test modules (e.g.test_openstack_env.py,test_openstack_baremetal.py,test_openstack_managers.py) to keep individual files easier to navigate and reason about. - The tests reach into
osism.tasks.conductor.utilsforget_vault,deep_decrypt, andmask_secretswhile primarily exercisingosism.tasks.openstack; if possible, isolating the masking behavior behind a smaller interface inopenstackand stubbing that instead would reduce coupling to the conductor implementation details and make the tests more robust to refactors.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `test_openstack.py` module is quite large (~1600 lines) and covers many distinct concerns (cloud env, connections, baremetal, NetBox, managers); consider splitting it into multiple test modules (e.g. `test_openstack_env.py`, `test_openstack_baremetal.py`, `test_openstack_managers.py`) to keep individual files easier to navigate and reason about.
- The tests reach into `osism.tasks.conductor.utils` for `get_vault`, `deep_decrypt`, and `mask_secrets` while primarily exercising `osism.tasks.openstack`; if possible, isolating the masking behavior behind a smaller interface in `openstack` and stubbing that instead would reduce coupling to the conductor implementation details and make the tests more robust to refactors.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
8183c4b to
2cd4821
Compare
|
|
||
| openstack_tasks.run_openstack_command_with_cloud("req-1", "/bin/x", "admin", []) | ||
|
|
||
| command_env.run.assert_called_once() |
There was a problem hiding this comment.
This asserts the OpenStack CLI is still launched when setup_cloud_environment returns cloud_setup_success=False, which locks in a real bug. run_openstack_command_with_cloud unpacks cloud_setup_success (osism/tasks/openstack.py:769) but never checks it, so it always calls run_command(..., ignore_env=True). ignore_env=True launches the CLI with an empty environment — no OS_* variables inherited — so its only credential source is the clouds.yaml/secure.yml files that setup_cloud_environment writes to /tmp on success. When setup failed (e.g. a cloud's password is missing and there's no secure.yml fallback), those files aren't there and the command runs with no credentials instead of aborting. Please fix the short-circuit in its own commit and flip this test to assert run is not called on setup failure. The same unread flag is at :805, :896, :936.
| cleanup = mocker.patch("osism.tasks.openstack.cleanup_cloud_environment") | ||
| run = mocker.patch("osism.tasks.openstack.run_command") | ||
|
|
||
| with pytest.raises(IndexError): |
There was a problem hiding this comment.
This asserts an IndexError as the contract, but that crash defeats the very guard it exercises. The configs branch strips any incoming --images so it can force --images <temp_dir> (osism/tasks/openstack.py:817-827); the --images X form pops twice but only guards ValueError, so a trailing bare --images pops past the end and crashes. Please harden the strip in its own commit — pop the value only if one follows — and assert the robust behavior here (trailing --images stripped, --images temp_dir forced, no crash) rather than pinning the crash. (No current caller passes --images into the configs branch, so this path isn't reachable via the CLI today; if the guard is considered unnecessary, removing it and dropping this test is the alternative.)
|
|
||
| def get_baremetal_node_parameters(node_uuid): | ||
| """Get kernel append params, netplan params, and FRR params for a node. | ||
| def _mask_node_secret_parameters( |
There was a problem hiding this comment.
This _mask_node_secret_parameters extraction is a production change to the module's surface but ships in the same commit as all the new tests, under an "Add unit tests..." subject. Per our commit-separation convention, please split it into its own commit (accurate subject) ahead of the test-addition commit. The extraction itself looks behavior-preserving.
Extract the secret-masking logic of get_baremetal_node_parameters into a dedicated _mask_node_secret_parameters helper, concentrating the dependency on osism.tasks.conductor.utils (get_vault, deep_decrypt, mask_secrets) behind a single seam. Callers -- and their tests -- no longer need to reach into conductor implementation details. Assisted-by: Claude:claude-fable-5 Assisted-by: Claude:claude-opus-4-8[1m] Signed-off-by: Christian Berendt <berendt@osism.tech>
run_openstack_command_with_cloud, the configs branch of image_manager, project_manager and project_manager_sync unpacked the success flag returned by setup_cloud_environment but never checked it. Because these commands run with ignore_env=True (an empty environment, no inherited OS_* variables), their only credential source is the clouds.yaml and secure.yml files that setup_cloud_environment writes to /tmp on success. When the setup failed -- for example because a cloud's password is missing and there is no secure.yml fallback -- the command was launched without any credentials instead of aborting. Short-circuit all four call sites: log an error and return 1 without running the command when the setup did not succeed. The cleanup still runs to remove anything a partially completed setup may have left behind. Assisted-by: Claude:claude-fable-5 Signed-off-by: Christian Berendt <berendt@osism.tech>
The configs branch of image_manager strips any incoming --images arguments so it can force --images <temp_dir>. The "--images <value>" form popped twice but only guarded ValueError, so a trailing bare --images without a value popped past the end of the list and crashed with an IndexError. Pop the value only if one follows the flag. Assisted-by: Claude:claude-fable-5 Signed-off-by: Christian Berendt <berendt@osism.tech>
Cover the cloud credential/environment helpers (get_cloud_password, setup_cloud_environment, cleanup_cloud_environment and the module-level get_openstack_connection), the baremetal/NetBox getters including the secret masking in get_baremetal_node_parameters, the thin Celery task wrappers around the OpenStack SDK, and the manager tasks (image_manager, flavor_manager, project_manager/project_manager_sync), including the short-circuit on failed cloud environment setup and the stripping of --images arguments in the image_manager configs branch. Split the suite across three modules by concern -- the cloud environment/connection helpers (test_openstack_env.py), the baremetal/NetBox getters and thin task wrappers (test_openstack_baremetal.py) and the manager tasks (test_openstack_managers.py) -- with the shared mock_os fixture in tests/unit/tasks/conftest.py. All external effects are mocked: the SDK connection comes from a patched osism.utils.get_openstack_connection, the NetBox client replaces the lazy osism.utils.nb attribute, and the environment helpers never touch the real filesystem or working directory because the module-level os/shutil/yaml name bindings (plus builtins.open) are patched. keystoneauth1 and openstacksdk are pinned runtime dependencies, so their real exception classes are raised instead of stand-ins. Assisted-by: Claude:claude-fable-5 Assisted-by: Claude:claude-opus-4-8[1m] Signed-off-by: Christian Berendt <berendt@osism.tech>
2cd4821 to
5f9e3be
Compare
Summary
Adds
tests/unit/tasks/test_openstack.pywith unit tests for the OpenStack Celery worker module, covering the targets listed in #2356:get_cloud_password(hyphen normalization, key-format guard, whitespace stripping, non-string coercion, loader errors),setup_cloud_environment(password-injection branch incl.clouds.yml/missing-auth/missing-profile variants,secure.yml/secure.yamlfallback branch, error paths),cleanup_cloud_environmentandget_cloud_helpers.get_openstack_connection(cloud, password): allUnauthorized/MissingRequiredOptions/SDKExceptionexit paths for both the password and the secure.yml attempt, the secure.yml retry after a failed password, and thecurrent_projectconnection probe. The realkeystoneauth1/openstacksdkexception classes are raised.get_baremetal_nodes,get_baremetal_node_ports,get_baremetal_node_netbox_info/get_baremetal_nodes_netbox_info(incl.cf_inventory_hostnamefallback), and the secret masking inget_baremetal_node_parameters(value-based masking,ironic_osism_*name-based regex masking after dropped vault decryption,secret_valuesforwarding intomask_secrets).baremetal_node_createattribute injection,wait_for_nodes_provision_statefirst/empty result,set_power_statewait/no-wait, generator materialization,set_target_raid_configmicroversion PUT).image_manager(delegation without configs, temp-dir config rewriting of--imagesarguments, cleanup on error, documentedIndexErrorfor a trailing--images),flavor_manager,project_manager/project_manager_sync, and the task-lock check for all four.Notes
MissingRequiredOptionswithSimpleNamespace(name=...); keystoneauth1 5.15.0 builds the message fromo.dest, so the tests useSimpleNamespace(dest=...).os.path.exists/os.chdirglobally, the tests patch the module-levelos/shutil/yamlname bindings ofosism.tasks.openstack, so no test can touch the real filesystem or working directory.Closes #2356
🤖 Generated with Claude Code