[tonyandrewmeyer]

Makes the broad charm compatibility tests actually run each charm's tests against the ops wheels. At the moment a lot of the jobs pass without ever doing that: in the September run, 20 of the 89 charm jobs installed the wheels with the system pip (tox-uv environments have no pip of their own, so pip install went to the runner's user site-packages), and others put the charm's locked ops back afterwards with poetry install or ran the tests through uv run, which uses the project's own environment.

• The wheels are installed with uv pip install, which goes into the tox environment via VIRTUAL_ENV, and with dependencies, since a charm locked to ops 2 doesn't have everything ops 3 needs (opentelemetry, for example).
• They're installed again after each poetry install or uv sync in the tox commands, not just the last one, since some charms run their unit tests between two installs.
• The ops-tracing wheel is only installed where the charm already has ops-tracing. Installing it everywhere would add the OpenTelemetry SDK to charms that don't use tracing, and leaving an older ops-tracing in place means it requires the old ops. This needs a small sh -c wrapper in the tox commands, so sh is added to allowlist_externals.
• For uv charms, ops, ops-scenario, and ops-tracing are overridden in pyproject.toml and the charm relocked, so anything installing from uv.lock gets the wheels. The override keeps the extras from the lock, including ones that only a charm library asks for.
• A new step fails the job if any environment the tests created has a different ops, ops-scenario, or ops-tracing version than the matching wheel, or if there's no ops installed at all, so any other way around the wheels shows up as a failure rather than a pass.
• The patch script has tests now (test/test_patch_charm_deps.py), since its only feedback so far has been the monthly run, and it went a long time not doing what it said without anyone noticing. This adds tomli-w to the unit dependency group, and the tests skip on Python 3.10, which doesn't have tomllib.

A run on my fork with an earlier version of this change (https://github.com/tonyandrewmeyer/operator/actions/runs/36358609881) has 11 jobs that passed in September now failing, all of them charms that weren't getting the wheels before: parca-k8s, zookeeper and zookeeper-k8s, temporal-k8s and temporal-ui, kafka-k8s, the tempo coordinator, both mysql-router charms (whose tests use the Scenario 6 API), and grafana-agent's static check. k8s-operator's static check now passes. That run predates the ops-tracing and install-after-each changes, so the list will probably shift a bit. I haven't gone through each of those yet to see which are real ops regressions and which are charms that need to update their tests, so the next scheduled run will be noticeably redder, and I'll work through them after this is merged.

In that run the new check only caught one charm, kafka-operator, which runs poetry run pip install -U ./common after the wheels are installed and so tests with ops 3.8.2. That one was already failing, and I've left it failing rather than special-casing it.

I found this while trialling canonical/hyrum as a replacement for this workflow, which relocks rather than reinstalling and so had been reporting failures here that this workflow wasn't.

[tonyandrewmeyer]

I'm not sure it's worth doing this, given that I really want to propose the other change, using hyrum. It would make for a better comparison, but I'm not sure that's worth bothering with.