---

Fast CPU inference for wav2vec2 ASR models — no Python, no PyTorch, no CUDA required.

whisper.cpp did this for Whisper. wav2vec2 has thousands of community fine-tuned models on HuggingFace — Telugu, Tamil, Arabic, Hindi, Swahili, Bangla, 100+ languages — and none of them had a C++ inference path until now.

---

Features

• ggml-powered — ships ggml as a submodule: quantized matmul, SIMD kernels (NEON / AVX2), and a clear upgrade path to Metal and CUDA
• Q8_0 and Q4_0 quantization — linear weights quantized at convert time; inference dequantizes on-the-fly. wav2vec2-base drops from 360 MB (F32) to 96 MB (Q8_0) or 48 MB (Q4_0)
• Universal — converts any Wav2Vec2ForCTC model from HuggingFace to GGUF
• Built-in WAV reader — no libsndfile, no miniaudio; just a file path
• Greedy + beam search CTC — greedy by default (-b 1), full prefix beam search with -b N
• Multi-threaded attention — transformer heads split across CPU cores (-t N)
• ARM + x86 — ggml auto-selects NEON / AVX2 / AVX512 at configure time
• Tested — 83 unit tests covering math ops, the GGUF reader, and the beam search decoder

Build

git clone https://github.com/py-ai-dev/wav2vec2.cpp
cd wav2vec2.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

Run tests:

cd build && ctest --output-on-failure

Requirements: C++17 compiler, CMake ≥ 3.14. ggml is a submodule and builds automatically — no other dependencies needed.

git clone --recursive https://github.com/py-ai-dev/wav2vec2.cpp

(If you cloned without --recursive: git submodule update --init)

Quick Start

1. Convert a model

pip install transformers torch
python scripts/convert_to_gguf.py facebook/wav2vec2-base-960h model.gguf          # F16 (default)
python scripts/convert_to_gguf.py facebook/wav2vec2-base-960h model_q8.gguf --dtype q8_0  # ~96 MB
python scripts/convert_to_gguf.py facebook/wav2vec2-base-960h model_q4.gguf --dtype q4_0  # ~48 MB

--dtype choices: f32, f16 (default), q8_0, q4_0. Quantization applies only to linear weight matrices; norms, biases, and conv weights stay in F32.

2. Transcribe

./build/wav2vec2-cli -m model.gguf -f audio.wav

usage: wav2vec2-cli -m MODEL -f AUDIO [-t THREADS] [-b BEAM] [-v]

  -m  model.gguf    GGUF model file
  -f  audio.wav     input WAV (16 kHz mono recommended; stereo/other rates auto-handled)
  -t  4             number of threads (default: 4)
  -b  1             CTC beam width: 1 = greedy, 5 = beam search (default: 1)
  -v                verbose output with timing

Examples

# English (360 MB base model)
python scripts/convert_to_gguf.py facebook/wav2vec2-base-960h      en.gguf
./build/wav2vec2-cli -m en.gguf -f speech.wav

# Telugu
python scripts/convert_to_gguf.py vasista22/wav2vec2-telugu-large   te.gguf
./build/wav2vec2-cli -m te.gguf -f telugu.wav

# Arabic (XLS-R fine-tune)
python scripts/convert_to_gguf.py jonatasgrosman/wav2vec2-large-xlsr-53-arabic  ar.gguf
./build/wav2vec2-cli -m ar.gguf -f arabic.wav

# Any other language — same pattern
python scripts/convert_to_gguf.py <any-wav2vec2-ctc-model> out.gguf
./build/wav2vec2-cli -m out.gguf -f audio.wav

Architecture

raw audio (16 kHz float32)
  ↓
CNN Feature Extractor     7 conv layers, total stride 320 → ~49 frames/sec
  ↓
Feature Projection        linear + layer norm  [T × conv_dim → T × hidden]
  ↓
Positional Conv Embed     grouped Conv1D, adds position information
  ↓
Transformer Encoder       12–24 layers, full bidirectional self-attention
  ↓
CTC Head                  linear → argmax over vocab → remove dups/blanks
  ↓
transcript

Supported Models

Any Wav2Vec2ForCTC checkpoint with:

Config field	Supported values
feat_extract_norm	"group" (base) or "layer" (large/XLS-R)
feat_extract_activation	"gelu"
Architecture	standard 7-layer CNN + transformer

This covers wav2vec2-base, wav2vec2-large, XLS-R-300M, XLS-R-1B, and the vast majority of community fine-tunes.

Performance

Tested on a 20-core ARM (Cortex-X925 + A725), 120 GB RAM:

Model	Size	10s audio	RTF
wav2vec2-base-960h	360 MB	~0.8 s	0.08
wav2vec2-large-xlsr	1.18 GB	~3.2 s	0.32

RTF < 1.0 = faster than real time. Both models run comfortably on CPU with no GPU.

Repository Structure

wav2vec2.cpp/
├── include/wav2vec2.h          public C API
├── src/
│   ├── ops.h                   math primitives (gelu, layer_norm, conv1d, …)
│   ├── gguf.h                  minimal GGUF reader
│   └── wav2vec2.cpp            model loading + forward pass
├── examples/main/main.cpp      CLI tool
├── tests/
│   ├── test_ops.cpp            unit tests — all math ops
│   └── test_gguf.cpp           unit tests — GGUF reader
├── scripts/convert_to_gguf.py  HuggingFace → GGUF converter
└── CMakeLists.txt

Roadmap

• Language model rescoring (KenLM)
• Python bindings
• WASM / browser support
• Android / iOS examples
• Batch inference
• Metal / CUDA backend

Contributing

PRs welcome. Please add or update tests for any changed logic. Run ctest before submitting.

License

MIT — see LICENSE.

Produced by liodon-ai.