Skip to content

Add unit tests for osism/tasks/openstack.py#2459

Merged
ideaship merged 4 commits into
mainfrom
unit-tests-tasks-openstack
Jul 17, 2026
Merged

Add unit tests for osism/tasks/openstack.py#2459
ideaship merged 4 commits into
mainfrom
unit-tests-tasks-openstack

Conversation

@berendt

@berendt berendt commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

Adds tests/unit/tasks/test_openstack.py with unit tests for the OpenStack Celery worker module, covering the targets listed in #2356:

  • Cloud credential/environment helpers: get_cloud_password (hyphen normalization, key-format guard, whitespace stripping, non-string coercion, loader errors), setup_cloud_environment (password-injection branch incl. clouds.yml/missing-auth/missing-profile variants, secure.yml/secure.yaml fallback branch, error paths), cleanup_cloud_environment and get_cloud_helpers.
  • Module-level get_openstack_connection(cloud, password): all Unauthorized/MissingRequiredOptions/SDKException exit paths for both the password and the secure.yml attempt, the secure.yml retry after a failed password, and the current_project connection probe. The real keystoneauth1/openstacksdk exception classes are raised.
  • Baremetal/NetBox getters: get_baremetal_nodes, get_baremetal_node_ports, get_baremetal_node_netbox_info / get_baremetal_nodes_netbox_info (incl. cf_inventory_hostname fallback), and the secret masking in get_baremetal_node_parameters (value-based masking, ironic_osism_* name-based regex masking after dropped vault decryption, secret_values forwarding into mask_secrets).
  • Thin Celery task wrappers: parametrized delegation block plus the specific branches (baremetal_node_create attribute injection, wait_for_nodes_provision_state first/empty result, set_power_state wait/no-wait, generator materialization, set_target_raid_config microversion PUT).
  • Manager tasks: image_manager (delegation without configs, temp-dir config rewriting of --images arguments, cleanup on error, documented IndexError for a trailing --images), flavor_manager, project_manager/project_manager_sync, and the task-lock check for all four.

Notes

  • The issue hint suggested constructing MissingRequiredOptions with SimpleNamespace(name=...); keystoneauth1 5.15.0 builds the message from o.dest, so the tests use SimpleNamespace(dest=...).
  • Instead of patching os.path.exists/os.chdir globally, the tests patch the module-level os/shutil/yaml name bindings of osism.tasks.openstack, so no test can touch the real filesystem or working directory.

Closes #2356

🤖 Generated with Claude Code

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The test_openstack.py module is quite large (~1600 lines) and covers many distinct concerns (cloud env, connections, baremetal, NetBox, managers); consider splitting it into multiple test modules (e.g. test_openstack_env.py, test_openstack_baremetal.py, test_openstack_managers.py) to keep individual files easier to navigate and reason about.
  • The tests reach into osism.tasks.conductor.utils for get_vault, deep_decrypt, and mask_secrets while primarily exercising osism.tasks.openstack; if possible, isolating the masking behavior behind a smaller interface in openstack and stubbing that instead would reduce coupling to the conductor implementation details and make the tests more robust to refactors.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `test_openstack.py` module is quite large (~1600 lines) and covers many distinct concerns (cloud env, connections, baremetal, NetBox, managers); consider splitting it into multiple test modules (e.g. `test_openstack_env.py`, `test_openstack_baremetal.py`, `test_openstack_managers.py`) to keep individual files easier to navigate and reason about.
- The tests reach into `osism.tasks.conductor.utils` for `get_vault`, `deep_decrypt`, and `mask_secrets` while primarily exercising `osism.tasks.openstack`; if possible, isolating the masking behavior behind a smaller interface in `openstack` and stubbing that instead would reduce coupling to the conductor implementation details and make the tests more robust to refactors.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@berendt
berendt force-pushed the unit-tests-tasks-openstack branch 2 times, most recently from 8183c4b to 2cd4821 Compare July 15, 2026 08:51
@berendt berendt moved this from New to Ready for review in Human Board Jul 15, 2026
@berendt
berendt requested a review from ideaship July 15, 2026 09:20
Comment thread tests/unit/tasks/test_openstack_env.py Outdated

openstack_tasks.run_openstack_command_with_cloud("req-1", "/bin/x", "admin", [])

command_env.run.assert_called_once()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This asserts the OpenStack CLI is still launched when setup_cloud_environment returns cloud_setup_success=False, which locks in a real bug. run_openstack_command_with_cloud unpacks cloud_setup_success (osism/tasks/openstack.py:769) but never checks it, so it always calls run_command(..., ignore_env=True). ignore_env=True launches the CLI with an empty environment — no OS_* variables inherited — so its only credential source is the clouds.yaml/secure.yml files that setup_cloud_environment writes to /tmp on success. When setup failed (e.g. a cloud's password is missing and there's no secure.yml fallback), those files aren't there and the command runs with no credentials instead of aborting. Please fix the short-circuit in its own commit and flip this test to assert run is not called on setup failure. The same unread flag is at :805, :896, :936.

cleanup = mocker.patch("osism.tasks.openstack.cleanup_cloud_environment")
run = mocker.patch("osism.tasks.openstack.run_command")

with pytest.raises(IndexError):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This asserts an IndexError as the contract, but that crash defeats the very guard it exercises. The configs branch strips any incoming --images so it can force --images <temp_dir> (osism/tasks/openstack.py:817-827); the --images X form pops twice but only guards ValueError, so a trailing bare --images pops past the end and crashes. Please harden the strip in its own commit — pop the value only if one follows — and assert the robust behavior here (trailing --images stripped, --images temp_dir forced, no crash) rather than pinning the crash. (No current caller passes --images into the configs branch, so this path isn't reachable via the CLI today; if the guard is considered unnecessary, removing it and dropping this test is the alternative.)

Comment thread osism/tasks/openstack.py

def get_baremetal_node_parameters(node_uuid):
"""Get kernel append params, netplan params, and FRR params for a node.
def _mask_node_secret_parameters(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This _mask_node_secret_parameters extraction is a production change to the module's surface but ships in the same commit as all the new tests, under an "Add unit tests..." subject. Per our commit-separation convention, please split it into its own commit (accurate subject) ahead of the test-addition commit. The extraction itself looks behavior-preserving.

@github-project-automation github-project-automation Bot moved this from Ready for review to In review in Human Board Jul 16, 2026
berendt added 4 commits July 16, 2026 18:11
Extract the secret-masking logic of get_baremetal_node_parameters into a
dedicated _mask_node_secret_parameters helper, concentrating the
dependency on osism.tasks.conductor.utils (get_vault, deep_decrypt,
mask_secrets) behind a single seam. Callers -- and their tests -- no
longer need to reach into conductor implementation details.

Assisted-by: Claude:claude-fable-5
Assisted-by: Claude:claude-opus-4-8[1m]
Signed-off-by: Christian Berendt <berendt@osism.tech>
run_openstack_command_with_cloud, the configs branch of image_manager,
project_manager and project_manager_sync unpacked the success flag
returned by setup_cloud_environment but never checked it. Because these
commands run with ignore_env=True (an empty environment, no inherited
OS_* variables), their only credential source is the clouds.yaml and
secure.yml files that setup_cloud_environment writes to /tmp on success.
When the setup failed -- for example because a cloud's password is
missing and there is no secure.yml fallback -- the command was launched
without any credentials instead of aborting.

Short-circuit all four call sites: log an error and return 1 without
running the command when the setup did not succeed. The cleanup still
runs to remove anything a partially completed setup may have left
behind.

Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@osism.tech>
The configs branch of image_manager strips any incoming --images
arguments so it can force --images <temp_dir>. The "--images <value>"
form popped twice but only guarded ValueError, so a trailing bare
--images without a value popped past the end of the list and crashed
with an IndexError. Pop the value only if one follows the flag.

Assisted-by: Claude:claude-fable-5
Signed-off-by: Christian Berendt <berendt@osism.tech>
Cover the cloud credential/environment helpers (get_cloud_password,
setup_cloud_environment, cleanup_cloud_environment and the module-level
get_openstack_connection), the baremetal/NetBox getters including the
secret masking in get_baremetal_node_parameters, the thin Celery task
wrappers around the OpenStack SDK, and the manager tasks (image_manager,
flavor_manager, project_manager/project_manager_sync), including the
short-circuit on failed cloud environment setup and the stripping of
--images arguments in the image_manager configs branch.

Split the suite across three modules by concern -- the cloud
environment/connection helpers (test_openstack_env.py), the
baremetal/NetBox getters and thin task wrappers
(test_openstack_baremetal.py) and the manager tasks
(test_openstack_managers.py) -- with the shared mock_os fixture in
tests/unit/tasks/conftest.py.

All external effects are mocked: the SDK connection comes from a patched
osism.utils.get_openstack_connection, the NetBox client replaces the
lazy osism.utils.nb attribute, and the environment helpers never touch
the real filesystem or working directory because the module-level
os/shutil/yaml name bindings (plus builtins.open) are patched.
keystoneauth1 and openstacksdk are pinned runtime dependencies, so their
real exception classes are raised instead of stand-ins.

Assisted-by: Claude:claude-fable-5
Assisted-by: Claude:claude-opus-4-8[1m]
Signed-off-by: Christian Berendt <berendt@osism.tech>
@berendt
berendt force-pushed the unit-tests-tasks-openstack branch from 2cd4821 to 5f9e3be Compare July 16, 2026 16:15
@berendt
berendt requested a review from ideaship July 16, 2026 16:32
@ideaship
ideaship merged commit 219ef44 into main Jul 17, 2026
3 checks passed
@ideaship
ideaship deleted the unit-tests-tasks-openstack branch July 17, 2026 09:28
@github-project-automation github-project-automation Bot moved this from In review to Done in Human Board Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Unit tests for osism/tasks/openstack.py

3 participants