test_ollama_model_names_exist false-fails for hf.co/ Ollama model references #1600

Description

test/backends/test_model_ids.py::test_ollama_model_names_exist[IBM_GRANITE_VISION_4_1_4B] fails with a false positive. The model ID is valid and intentional, but the test's registry check cannot handle the hf.co/ Ollama convention.

Steps to Reproduce

uv run pytest test/backends/test_model_ids.py -m "not qualitative"

Output:

FAILED test/backends/test_model_ids.py::test_ollama_model_names_exist[IBM_GRANITE_VISION_4_1_4B]
Failed: IBM_GRANITE_VISION_4_1_4B.ollama_name='hf.co/ibm-granite/granite-vision-4.1-4b-GGUF:Q4_K_M'
does not exist in the Ollama library. Update or remove the model ID in mellea/backends/model_ids.py.

Full traceback shows the failure path:

client.show(ollama_name) falls through because the model is not cached locally.
The test constructs the registry URL:
https://registry.ollama.ai/v2/hf.co/ibm-granite/granite-vision-4.1-4b-GGUF/manifests/Q4_K_M

urllib.request.urlopen(req, timeout=10) raises HTTP Error 404: Not Found.
The e.code in (404, 401) branch calls pytest.fail(...).

Environment: Python 3.12.5, pytest 9.0.3, macOS darwin arm64 — 1 failed, 93 passed in 45.25s.

Root Cause

IBM_GRANITE_VISION_4_1_4B in mellea/backends/model_ids.py sets:

ollama_name="hf.co/ibm-granite/granite-vision-4.1-4b-GGUF:Q4_K_M"

The hf.co/<repo> prefix is a special Ollama model reference that allows Ollama to fetch a GGUF model directly from Hugging Face rather than resolving it through the Ollama registry.

These references are intentionally not registered at registry.ollama.ai.

The test currently validates every non-local ollama_name by issuing a HEAD request to:

https://registry.ollama.ai/v2/{name}/manifests/{tag}

For an hf.co/ reference, this registry lookup fails because the model is not expected to exist in the Ollama registry. The test interprets the resulting 404 or 401 as evidence that the model ID is invalid and calls pytest.fail(), producing a false positive.

The existing comment on the constant already documents this behavior:

"Not in the Ollama library; the hf.co/... tag pulls IBM's official GGUF build straight from Hugging Face."

Additionally, IBM_GRANITE_VISION_4_1_4B.ollama_name is actively used by test/backends/test_vision_ollama.py, so setting ollama_name to None is not an option.

Proposed Fix

Skip the Ollama registry check for hf.co/ references, since they cannot be validated through registry.ollama.ai by design:

if ollama_name.startswith("hf.co/"):
    pytest.skip(
        f"{const_name}.ollama_name={ollama_name!r} is an hf.co/ reference "
        "resolved directly from Hugging Face rather than the Ollama registry; "
        "skipping registry validation."
    )

This preserves the existing behavior:

• Models available locally are validated using client.show().
• Standard Ollama registry models are validated against registry.ollama.ai.
• hf.co/ references are excluded from registry validation because they use a different resolution mechanism.

Affected Files
test/backends/test_model_ids.py — add an hf.co/ carve-out before the registry check.
mellea/backends/model_ids.py — no change required; the existing model ID is intentional and correctly documented.