Reproducible model conversion: Hugging Face → GGUF/PTQTP with quality gate

MLX loads Hugging Face safetensors directly; here a model needs a conversion. For a platform runner that must be a pipeline, not a manual step.

tool: HF checkpoint → GGUF / PTQTP at a pinned revision, recorded hashes
quality gate: reference suite (the one geist-serve uses for its verdicts) and perplexity against the source model
output: a manifest entry for geist-runtime (Model manifest, catalog and verification geist-runtime#5)

Done when

one command converts a pinned HF revision; two runs produce byte-identical output (hashes recorded)
quality gate: reference-suite score at most 2 points below the source model at the same precision, perplexity delta within a documented bound per quantization
it works for at least two families (Gemma, Qwen) and one small model runs in CI
a manifest entry for geist-runtime (ci: ASan + UBSan unit-test job #5) is generated, not hand-written

Plus the common definition of done in geisten/geist-serve#149 and the roadmap gates (parent baseline, no regression).

Merge pull request #676 from geisten/feat/623-convert-hf

Done in #676 (tools/convert_hf.py: convert / gate / manifest) and #678 (the convert-gate workflow builds with gcc-14).

Pipeline: a pinned Hugging Face revision goes through llama.cpp's converter at the cross-engine pin, optionally followed by llama-quantize. The output gets a manifest recording every input's sha256, the converter commit and the Python deps. --check-determinism converts twice and fails unless the outputs are byte-identical.
Quality gate: perplexity against the source precision through eval_geist, on a fixed public-domain text, with the model's BOS policy. Bounds are set per quantization and are wider below 2 B parameters.
CI: the convert-gate workflow runs SmolLM2 135M end to end, both weekly and on changes. It passed on convert-gate: build with gcc-14 #678.
Verified on Qwen3 0.6B (BF16 / Q8_0 / Q4_K_M), Gemma 4 E2B (BF16 / Q8_0) and Gemma 3 270M (BF16 / Q8_0 +0.09 %).
Found while building it: transformer: Gemma 3 270M gives wrong logits (perplexity 133375 vs llama.cpp 45 on the same GGUF) #674 (Gemma 3 logits, fix in gemma3: sliding-window layers, query_pre_attn_scalar, no V norm (#674) #677) and gguf: Q5_0 weights are unsupported — llama-quantize's Q4_K_M of small models does not load #675 (Q5_0 weights).

Generated by Claude Code

Navigation Menu