Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 2 additions & 21 deletions docs/explanation/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ When writing an integration test, it is not sufficient to simply check that Juju
### Tools

- [`pytest`](https://pytest.org/) or [`unittest`](https://docs.python.org/3/library/unittest.html) and
- [pytest-operator](https://github.com/charmed-kubernetes/pytest-operator)
- [Jubilant](https://documentation.ubuntu.com/jubilant/)

Integration tests and unit tests should run using the minor version of Python that is shipped with the OS specified in `charmcraft.yaml` (the `base.run-on` key). For example, if Ubuntu 22.04 is specified in `charmcraft.yaml`, you can use the following tox configuration:

Expand All @@ -121,28 +121,9 @@ Integration tests and unit tests should run using the minor version of Python th
basepython = python3.10
```

(pytest-operator)=
#### `pytest-operator`

`pytest-operator` is a Python library that provides Juju plugins for the generic Python library `pytest` to facilitate the {ref}`integration testing <integration-testing>` of charms.

> See more: [`pytest-operator`](https://github.com/charmed-kubernetes/pytest-operator)

It builds a fixture called `ops_test` that helps you interact with Juju through constructs that wrap around [`python-libjuju` ](https://pypi.org/project/juju/).

> See more:
> - [`pytest-operator` > `ops_test`](https://github.com/charmed-kubernetes/pytest-operator/blob/main/docs/reference.md#ops_test)
> - [`pytest` > Fixtures](https://docs.pytest.org/en/6.2.x/fixture.html)

It also provides convenient markers and command line parameters (e.g., the `@pytest.mark.skip_if_deployed` marker in combination with the `--no-deploy` configuration helps you skip, e.g., a deployment test in the case where you already have a deployment).

> See more:
> - [`pytest-operator` > Markers](https://github.com/charmed-kubernetes/pytest-operator/blob/main/docs/reference.md#markers)
> - [`pytest-operator` > Command line parameters](https://github.com/charmed-kubernetes/pytest-operator/blob/main/docs/reference.md#command-line-parameters)

### Examples

- [Prometheus K8s integration tests](https://github.com/canonical/prometheus-k8s-operator/blob/main/tests/integration/test_charm.py)
- [Tempo worker integration tests](https://github.com/canonical/tempo-operators/blob/main/worker/tests/integration/test_deploy.py)

## Continuous integration

Expand Down
13 changes: 5 additions & 8 deletions docs/howto/manage-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,14 +161,11 @@ def test_backup_action_failed():

> See first: {ref}`write-integration-tests-for-a-charm`

To verify that an action works correctly against a real Juju instance, write an integration test with `pytest_operator`. For example:
To verify that an action works correctly against a real Juju instance, write an integration test with `jubilant`. For example:

```python
async def test_logger(ops_test):
app = ops_test.model.applications[APP_NAME]
unit = app.units[0] # Run the action against the first unit.
action = await unit.run_action('snapshot', filename='db-snapshot.tar.gz')
action = await action.wait()
assert action.status == 'completed'
assert action.results['snapshot-size'].isdigit()
def test_logger(juju: jubilant.Juju):
action = juju.run("your-app/0", "snapshot", {"filename": "db-snapshot.tar.gz"})
assert action.status == "completed"
assert action.results["snapshot-size"].isdigit()
```
23 changes: 9 additions & 14 deletions docs/howto/manage-leadership-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,19 +85,14 @@ More commonly, an integration test might want to verify that leader and non-lead
as expected. For example:

```python
async def get_leader_unit(ops_test, app, model=None):
"""Utility method to get the current leader unit."""
leader_unit = None
if model is None:
model = ops_test.model
for unit in model.applications[app].units:
if await unit.is_leader_from_status():
leader_unit = unit
break

return leader_unit
def get_leader_unit(juju: jubilant.Juju) -> str | None:
"""Utility method to get the name of the current leader."""
for unit_name, unit in juju.status().apps["your-app"].units.items():
if unit.leader:
return unit_name
# It's possible that no leader has been elected,
# for example if the application has just been deployed.
return None
```

> Examples: [Zookeeper testing upgrades](https://github.com/canonical/zookeeper-operator/blob/106f9c2cd9408a172b0e93f741d8c9f860c4c38e/tests/integration/test_upgrade.py#L22), [postgresql testing password rotation action](https://github.com/canonical/postgresql-k8s-operator/blob/62645caa89fd499c8de9ac3e5e9598b2ed22d619/tests/integration/test_password_rotation.py#L38)

> See more: [`juju.unit.Unit.is_leader_from_status`](https://pythonlibjuju.readthedocs.io/en/latest/api/juju.unit.html#juju.unit.Unit.is_leader_from_status)
> See more: [](jubilant.Juju.status)
21 changes: 7 additions & 14 deletions docs/howto/manage-opened-ports.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,6 @@ By adding the following test to your `tests/integration/test_charm.py` file, you
that your charm opens a port specified in the configuration, but prohibits using port 22:

```python
async def get_address(ops_test: OpsTest, app_name: str, unit_num: int = 0) -> str:
"""Get the address for service for an app."""
status = await ops_test.model.get_status()
return status['applications'][app_name].public_address

def is_port_open(host: str, port: int) -> bool:
"""Check if a port is opened in a particular host."""
try:
Expand All @@ -74,26 +69,24 @@ def is_port_open(host: str, port: int) -> bool:
except (ConnectionRefusedError, TimeoutError):
return False # If connection fails, the port is closed

@pytest.mark.abort_on_fail
async def test_open_ports(ops_test: OpsTest):

def test_open_ports(juju: jubilant.Juju):
"""Verify that setting the server-port in the charm's opens that port.

Assert blocked status in case of port 22 and active status for others.
"""
app = ops_test.model.applications.get('my-charm')

# Get the public address of the app:
address = await get_address(ops_test=ops_test, app_name=APP_NAME)
address = juju.status().apps["your-app"].units["your-app/0"].public_address
# Validate that initial port is opened:
assert is_port_open(address, 8000)

# Set the port to 22 and validate the app goes to blocked status with the port not opened:
await app.set_config({'server-port': '22'})
await ops_test.model.wait_for_idle(apps=[APP_NAME], status='blocked', timeout=120)
juju.config("your-app", {"server-port": "22"})
juju.wait(jubilant.all_blocked)
assert not is_port_open(address, 22)

# Set the port to 6789 and validate the app goes to active status with the port opened.
await app.set_config({'server-port': '6789'})
await ops_test.model.wait_for_idle(apps=[APP_NAME], status='active', timeout=120)
juju.config("your-app", {"server-port": "6789"})
juju.wait(jubuilant.all_active)
assert is_port_open(address, 6789)
```
32 changes: 10 additions & 22 deletions docs/howto/manage-relations.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ requires:

Note that implementing a cross-model relation is done in the same way as one between applications in the same model. The ops library does not distinguish between data from a different model or from the same model as the one the charm is deployed to.

Which side of the relation is the 'provider' or the 'requirer' is sometimes arbitrary, but if one side has a workload that is a server and the other a client, then the server side should be the provider. This becomes important for how Juju sets up network permissions in cross-model relations.
Which side of the relation is the "provider" or the "requirer" is sometimes arbitrary, but if one side has a workload that is a server and the other a client, then the server side should be the provider. This becomes important for how Juju sets up network permissions in cross-model relations.
Comment thread
dimaqq marked this conversation as resolved.

If the relation is with a subordinate charm, make sure to set the `scope` field to `container`.

Expand Down Expand Up @@ -346,29 +346,17 @@ relation.remote_unit_name # 'zookeeper/42'

### Write integration tests

Write an integration test that verifies that your charm behaves as expected in a real Juju environment. Other than when testing peer relations, as well as deploying your own charm, the test needs to deploy a second application, either the real charm or a facade that provides enough functionality to test against.
> See first: {ref}`write-integration-tests-for-a-charm`

The pytest-operator plugin provides methods to deploy multiple charms. For example:
To verify that charm behaves correctly when integrated with another in a real Juju environment, write an integration test with `jubilant` that deploys another application and relates your charm to it.

```python
# This assumes that your integration tests already include the standard
Comment thread
dimaqq marked this conversation as resolved.
# build and deploy test that the Charmcraft profile provides.

@pytest.mark.abort_on_fail
async def test_active_when_deploy_db_facade(ops_test: OpsTest):
await ops_test.model.deploy('facade')
await ops_test.model.integrate(APP_NAME + ':postgresql', 'facade:provide-postgresql')

present_facade('postgresql', model=ops_test.model_name,
app_data={
'credentials': 'secret:abc',
})

await ops_test.model.wait_for_idle(
apps=[APP_NAME],
status='active',
timeout=600,
)
```
# build and deploy test.

> See more: [`pytest-operator`](https://pypi.org/project/pytest-operator/)
def test_active_with_another_app(juju: jubilant.Juju):
juju.deploy("another-app")
juju.integrate("your-app:endpoint", "another-app:endpoint")

juju.wait(jubilant.all_active)
```
15 changes: 5 additions & 10 deletions docs/howto/manage-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,17 +231,12 @@ ctx.run(ctx.on.storage_attached(foo_1), testing.State(storages={foo_0, foo_1}))

> See first: {ref}`write-integration-tests-for-a-charm`

To verify that adding and removing storage works correctly against a real Juju instance, write an integration test with `pytest_operator`. For example:
To verify that adding and removing storage works correctly against a real Juju instance, write an integration test with `jubilant`. For example:

```python
# This assumes there is a previous test that handles building and deploying.
async def test_storage_attaching(ops_test):
# Add a 1GB "cache" storage:
await ops_test.model.applications[APP_NAME].units[0].add_storage("cache", size=1024*1024)

await ops_test.model.wait_for_idle(
apps=[APP_NAME], status="active", timeout=600
)

def test_storage_attaching(juju: jubilant.Juju):
# Add two storage units of 2 gigabyte each to unit 0 of the Kafka app.
juju.cli("add-storage", "kafka/0", "data=2G,2", include_model=True)
juju.wait(jubilant.all_active)
# Assert that the storage is being used appropriately.
```
37 changes: 7 additions & 30 deletions docs/howto/manage-the-charm-version.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,42 +54,19 @@ uploading a charm to CharmHub (or when deploying/refreshing for local charms).

## Test the feature

Since the version isn't set by the charm code itself, you'll want to test that
Since the version isn't set by the charm code itself, you may want to test that
the version is correctly set with an integration test, and don't need to write
a unit test.

> See first: {ref}`write-integration-tests-for-a-charm`

To verify that setting the charm version works correctly in an integration test,
in your `tests/integration/test_charm.py` file, add a new test after the
`test_build_and_deploy` one that `charmcraft init` provides. In this test, get
the status of the model, and check the `charm_version` attribute of the unit.
For example:

```python
# `charmcraft init` will provide this test for you.
async def test_build_and_deploy(ops_test: OpsTest):
# Build and deploy charm from local source folder
charm = await ops_test.build_charm(".")

# Deploy the charm and wait for active/idle status
await asyncio.gather(
ops_test.model.deploy(charm, application_name=APP_NAME),
ops_test.model.wait_for_idle(
apps=[APP_NAME], status="active", raise_on_blocked=True, timeout=1000
),
)

async def test_charm_version_is_set(ops_test: OpsTest):
# Verify that the charm version has been set.
status = await ops_test.model.get_status()
version = status.applications[APP_NAME].charm_version
in your `tests/integration/test_charm.py` file, add a new test:

```py
def test_charm_version_is_set(juju: jubilant.Juju):
"""Verify that the charm version has been set."""
version = juju.status().apps["your-app"].charm_version
expected_version = subprocess.check_output(["git", "rev-parse", "HEAD"]).decode("utf8")
assert version == expected_version
```

<!---
No "see more" link: this is not currently documented in the pylibjuju docs.
-->

> Examples: [synapse checking that the unit's workload version matches the one reported by the server](https://github.com/canonical/synapse-operator/blob/778bcd414644c922373d542a304be14866835516/tests/integration/test_charm.py#L139)
7 changes: 1 addition & 6 deletions test/smoke/test_smoke.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,7 @@


def pack(charm_dir: pathlib.Path):
"""Pack the charm.

The pytest-operator plugin has a pack method, but it doesn't work out of the
box in GitHub actions, and there isn't really any reason that it should be
part of the plugin, so we just have a simple subprocess here.
"""
"""Pack the charm."""
cmd = ['charmcraft', 'pack', '--verbose']
# We need to use `sudo` in the GitHub actions environment, just as in
# the pack test. `sg lxd -c` should work, but does not - perhaps because of
Expand Down