composer: composed checkpoints emit torch_dtype: null, which can make vLLM resolve float32 #126

Where it lives: main. Found while working on feature/granite-turboctc-default (ef43e69).

arch.py:304 sets "torch_dtype": None in the architecture descriptor, so a composed config.json carries:

{ "torch_dtype": null, "dtype": "bfloat16" }

That split is the transformers-5 field rename. PR #116 documented the consequence in docs/SUPPORTED_MODELS.md:

--dtype bfloat16 — The composed config.json carries torch_dtype: null, so vLLM's default auto resolves to float32, and ~22 B params in fp32 is ~88 GB — an OOM that reads as a memory-sizing problem and is not one.

The behaviour is inconsistent, which is why this needs settling rather than documenting. A 4.2-3b engine on vLLM 0.20.2 logged dtype=torch.bfloat16 and resolved correctly despite torch_dtype: null; PR #116 observed a float32 fallback on a granitemoe base. So it is version- or path-dependent, and today we compensate with a --dtype bfloat16 flag in every deployment — a workaround in every serve command rather than a fix in one place.

Fix: write torch_dtype alongside dtype when composing, or establish that null is correct and record why, so the serve-flag requirement is a deliberate contract rather than a trap. Either way --dtype bfloat16 should stop being load-bearing.