Model Modernizer (CLI)

Model Modernizer is a production-ready Python CLI tool that detects a model type and converts it into a more modern/optimized format:

• LLM (Hugging Face) → GGUF (then quantize to q4_0)
• Stable Diffusion (Diffusers/HF) → Diffusers saved with .safetensors
• Whisper (Transformers/HF) → ONNX

Installation

Create a venv and install dependencies:

cd model-modernizer
python -m venv .venv
source .venv/bin/activate  # Linux/macOS
# .venv\Scripts\activate   # Windows PowerShell
pip install -r requirements.txt

Usage

Command:

python convert.py <model_path_or_hf_repo> <output_name>

Example (Hugging Face repo):

python convert.py mistralai/Mistral-7B-v0.1 mistral

Expected output style:

[INFO] Downloading model...
[INFO] Detected: LLM
[INFO] Converting to GGUF...
[INFO] Quantizing (q4)...
[SUCCESS] Model ready at output/mistral-q4.gguf

Outputs are written to output/ and logs to:

• logs/success.log
• logs/error.log

Model Detection

Detection logic is intentionally heuristic and fast:

• LLM: folder contains config.json with "architectures"
• Diffusion: file/folder names contain "diffusion", "unet", or "vae"
• Whisper: filename contains "whisper" (also checks config markers)
• Fallback: if unknown, it tries pipelines in order: LLM → diffusion → whisper

LLM Conversion: HF → GGUF (llama.cpp required)

This tool calls llama.cpp’s conversion script and quantizer:

1. convert-hf-to-gguf.py (HF → GGUF f16)
2. quantize / quantize.exe (GGUF → q4_0)

Install llama.cpp

Clone and build llama.cpp (follow upstream instructions for your OS). Then point Model Modernizer to your llama.cpp checkout/binaries using environment variables:

• LLAMA_CPP_DIR: path to llama.cpp folder (the tool will try common locations)
• or set explicit paths:
  • LLAMA_CPP_CONVERT: full path to convert-hf-to-gguf.py
  • LLAMA_CPP_QUANTIZE: full path to quantize / quantize.exe

Example (PowerShell):

$env:LLAMA_CPP_DIR="D:\tools\llama.cpp"
python convert.py mistralai/Mistral-7B-v0.1 mistral

Diffusion Conversion (Stable Diffusion)

Uses diffusers.StableDiffusionPipeline.from_pretrained() and saves using:

• save_pretrained(..., safe_serialization=True)

Output is a directory:

• output/<name>-diffusers/ containing one or more .safetensors files.

Whisper Conversion (ONNX)

Uses transformers.WhisperForConditionalGeneration and exports an ONNX graph via torch.onnx.export.

Output:

• output/<name>.onnx

Advanced Features

• Auto-download: If input looks like org/name, the tool downloads it from Hugging Face into models/.
• Retry logic: If the detected pipeline fails, it automatically tries the remaining pipelines.
• Validation: Outputs are checked for existence and basic sanity (file size / presence of .safetensors).
• Clean errors: Errors are printed to terminal and always written to logs/error.log.

Limitations

• LLM → GGUF requires a working llama.cpp toolchain on your machine.
• Whisper → ONNX exports a generic seq2seq forward; some deployments may prefer separate encoder/decoder exports or additional optimization.
• Some model repos require authentication; set HF_TOKEN if needed.