VibeVoice-ASR-BitNet

Microsoft Research's quantized edge-CPU ASR model, published on Hugging Face under the MIT licence. The card describes it as "a compressed variant of VibeVoice-ASR optimized for real-time inference on edge CPUs — no GPU required," and it is the first STT model in this spoke whose headline claim is a quantization recipe rather than an architecture.

The footprint

1.58 GB total, "2.9× compression from FP16" against a 4.62 GB uncompressed original, split across two components quantized differently:

I2_S is the ternary weight format from the BitNet line, which is where the repo name comes from — but only the decoder's weights use it, alongside Q6_K, and the tokenizer stays at 8-bit. The card body never uses the words "BitNet," "1-bit," or "ternary" anywhere, so the name describes one component's format, not the model. This is mixed-precision compression, not a 1.58-bit model, and the "1.58 GB" total is a coincidence of size, not the 1.58-bits-per-weight figure BitNet is known for. Worth stating plainly because the two numbers invite exactly that confusion.

It runs under llama.cpp (llama serve -hf microsoft/VibeVoice-ASR-BitNet:Q6_K), which puts it in the same delivery channel as whisper.cpp — GGUF-style artifacts served by a ggml runtime rather than a Python stack.

Speed

Measured on an AMD EPYC 7V13, real-time factor by thread count:

It crosses real-time (RTF < 1) at 3 threads, where Microsoft claims 1.86× faster than whisper.cpp, and reaches 0.42 at 8. The stated requirement is "RTF < 1 with 3+ threads on commodity x86 (AVX2) and ARM (NEON) hardware." Self-reported, single machine, no audio length or model-size pairing given for the whisper.cpp baseline — so the 1.86× is a vendor number. (Re-checked 2026-09-23: the card has not changed since 2026-07-24 — same table, MIT licence — and the model has ~44k downloads.)

Accuracy, and where quantization hurts

WER %, as published (lower is better; VibeVoice-ASR-7B is the uncompressed parent):

Two readings the table supports, both new for this spoke.

Quantization damage is uneven, and the pattern is legible. Most benchmarks lose 0.2–0.6 WER points going 7B → BitNet, which is the "quantization is nearly free" story whisper-cpp and quantization already tell. But AISHELL4 loses 7.6 points (19.83 → 27.45), AliMeeting 4.4, and the Korean and Vietnamese MLC sets 1.3 and 2.2. The heaviest losses are the far-field Chinese meeting sets and the lower-resource languages, so the compressed model's degradation lands hardest exactly where the task is hardest and the training data thinnest. A single average WER would have hidden that entirely.

It wins on hard audio and loses on clean audio. Against Whisper it is far ahead on meeting and far-field speech (AMI-ihm 21.36 vs 27.07, AMI-sdm 25.87 vs 36.92, MLC-EN 8.25 vs 13.57, VoxPopuli 5.18 vs 7.19) while losing on read speech (Libri-clean 2.41 vs 1.98, Fleurs-en 5.21 vs 3.99). Parakeet beats both on the clean sets by a wide margin (Libri-clean 1.49, Libri-other 3.13) and loses to this model on every meeting set shown. Whichever model you call "most accurate" is decided by which half of the table you read — the WER-leaderboard framing on speech-to-text and open-source-stt-models flattens a split that matters more than the ranking does.

Languages: English, Chinese, French, Italian, Korean, Portuguese, Vietnamese (7 named).

Contradiction: the parameter count

The card's model-size field reads "0.3B params" while the same card states a 4.62 GB FP16 original and calls the model a compressed variant of VibeVoice-ASR-7B. Those three numbers can't all be right: 0.3B parameters at FP16 is ~0.6 GB, not 4.62 GB, and 4.62 GB at FP16 is ~2.3B parameters, not 7B either. Recorded as published, unresolved. The 7B parent's own footprint is never stated.

Also unstated: training data, limitations, streaming or long-form behaviour, diarization, and any relationship to the VibeVoice TTS model Microsoft ships under the same name — this card documents only the ASR line.

Related

speech-to-text · whisper · whisper-cpp · moonshine · vosk · open-source-stt-models · quantization