OmniVoice

Zero-shot voice-cloning TTS from k2-fsa — Daniel Povey's group, whose ZipVoice this spoke already reaches through luxtts. Apache-2.0, Python, 9,891★ / 1,606 forks / 58 open issues, created 31 March 2026, last pushed 31 August 2026 (read 2026-09-05). Paper: Zhu, Ye, Kang, Yao, Guo, Kuang, Han, Zhuang, Lin & Povey, OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models, arXiv:2604.00688, v1 1 April 2026. Arrived by Telegram.

0.8B parameters, initialised from Qwen3-0.6B, generating 8 codebooks of Higgs-audio acoustic tokens. The architectural claim is that it skips the usual two-stage text→semantic→acoustic pipeline and maps text straight to multi-codebook acoustic tokens, trained with full-codebook random masking. Modes: voice cloning from a reference clip, voice design from described attributes (gender, age, pitch, accent, dialect, whisper), auto voice, plus [laughter] and pinyin/CMU pronunciation overrides.

Why this page is T3

Its load-bearing claims are benchmark scores the authors ran on their own model, which is what T3 describes (../QUALITY.md → Tier by the claim, not the URL). The repository metadata and the parameter count are artifact facts and would be T1 on their own; the reason the source is in the wiki is the performance and coverage claims, and nobody outside k2-fsa has checked any of them.

The 646-language claim, and what is under it

The README says "600+ languages" and the docs table says 646 languages, 581k hours, curated from 50 open-source datasets. The paper deserves credit for publishing what the headline hides, and the spoke should record both halves.

The per-language hours are in the docs, and the spread is roughly 5.6 million to one.

The paper names this itself — "severe data imbalance" — and resamples with β=0.8 to upsample the tail. The docs table does not: it lists all 646 rows identically, with an hours column and no quality tier, no support level and no caveat. A reader scanning for their language finds it present and finds nothing telling them that "present" can mean seventy-two seconds of audio.

And the evaluation covers a fraction of the list. The multilingual benchmarks are FLEURS-102 and MiniMax-Multilingual-24. That is at most 102 of 646 languages with any published number, so roughly 544 have a support claim and no measurement of any kind — not a bad one, none. This is the spoke's standing complaint in an unusually clean form: the coverage number is the product's headline, and the coverage number is the part nobody measured.

The numbers it does publish

Every figure below is the authors' own.

LibriSpeech-PC (English zero-shot): SIM-o 0.729, WER 1.30%, UTMOS 4.28 — against MaskGCT (0.691 / 2.26% / 3.91) and F5-TTS (0.655 / 1.89% / 3.89).

It beats the ground truth on that board, which is the line to read carefully rather than to repeat: ground-truth recordings score SIM-o 0.690, WER 1.87 and UTMOS 4.10. A synthetic voice "outscoring" real speech on WER and a MOS predictor is the known artifact of synthesised audio being cleaner and more canonically pronounced than a real recording of a real person in a real room. It says the metrics have run out of headroom, not that the model has passed human speech. The spoke has the same caution on mean-opinion-score and tts-benchmarks.

Seed-TTS: English SIM-o 0.741 / WER 1.60% / UTMOS 3.91; Mandarin SIM-o 0.777 / WER 0.84% / UTMOS 3.11.

MiniMax-Multilingual-24: average WER 2.850%, SIM-o 0.830, against MiniMax-Speech 3.774% and ElevenLabs v2 at 10.950%.

FLEURS-102: average CER 4.00% against a ground truth of 5.11%; 82 languages at CER ≤5% (ground truth 75) and 95 at ≤10% (ground truth 92). Same caveat as above, and note what the second pair of numbers actually says — on this board the recordings only clear 5% CER in 75 of 102 languages, so the measuring instrument is a large part of what is being reported.

Subjective: CMOS 0.44±0.16 (Qwen3-TTS 0.40±0.16), SMOS 3.80±0.17.

Speed: RTF 0.0319 at 16 sampling steps, 0.0598 at 32, batch size 1 — against ZipVoice's 0.0557, their own earlier model. The README's headline is RTF 0.025 and a 2.0–2.6× FlashInfer speedup with batching. Training: 2M updates in 9.66 days on 8× H800.

The limitations the paper states

Worth keeping, because a maker publishing failure modes is the part of a T3 source that carries the most weight:

• Trained only on open-source data, with "inconsistent annotation quality and uneven acoustic quality".
• Instruction-following is bounded by the diversity of available instruction-tuning data — and voice design was trained mainly on Chinese and English, so the described-attribute mode is not a 646-language feature.
• Not optimised for complex numeric or mathematical sequences.
• No inference-acceleration technique exists for discrete-space NAR TTS comparable to what continuous-space models get.
• Cantonese scores WER 17.71% under Whisper and 2.27% under SenseVoice-Small. The authors read that as the evaluator failing rather than the synthesis, and they are almost certainly right — which makes it the clearest instance in this spoke of ASR-scored TTS measuring the ASR. A 15-point swing from changing the judge is worth more than most of the table above it.

Related

text-to-speech · voice-cloning · open-weight-tts · tts-benchmarks · luxtts · fleurs · mean-opinion-score · sensevoice · xtts-paper · open-source-tts-models · speech-audio-ai · synthesis