Speech-to-text (STT / ASR)

Automatic speech recognition — transcribing speech audio into text. The recognition branch of speech-audio-ai (the mirror of text-to-speech‘s synthesis).

How it’s measured

WER (word error rate) — the headline accuracy metric; lower is better. The neutral anchor is Hugging Face’s Open ASR Leaderboard (since 2023; 700K+ visits; recently added a “benchmaxxer repellant” to resist overfitting). 2026 English leaders: canary-qwen 5.63%, IBM Granite Speech 5.85%, whisper V3 7.4% open-source-stt-models.
RTFx (real-time factor) — throughput; how many ×-real-time it transcribes. NVIDIA Parakeet TDT >2,000× (streaming, RNN-Transducer); Whisper V3 Turbo 216×.
Latency — for streaming/agents: Deepgram <300ms, ElevenLabs Scribe ~150ms stt-apis-comparison.
Language coverage — Whisper 99+, Google Chirp 125+, Qwen3-ASR 52.

The leaderboard stopped deciding things (2026-07)

The WER-ranking frame above has been overtaken twice in one month, from opposite directions.

From the top: MarkTechPost’s July roundup finds the top ten open models inside one WER point, with the board reordering when Appen’s private-track data is toggled on — and canary-qwen, recorded here in June as the leader at 5.63%, now sits behind ARK-ASR-3B (5.04), Granite Speech 4.1 (5.33), Cohere Transcribe (5.42) and MOSS-Transcribe-preview. The reported WERs aren’t even measured on the same set count (8 sets, 7 with TED-LIUM dropped, LibriSpeech-only). What decides a pick instead: licence — Apache-2.0 versus the CC-BY-4.0 attached to Canary-Qwen, Parakeet and Kyutai — plus language coverage, whether the architecture streams at all, and cost per audio-hour. Two more axes arrive as products rather than research: diarization (MOSS-Transcribe-Diarize 0.9B) and non-autoregressive decoding (Granite 2B-NAR). Language coverage gained an extreme: Meta’s Omnilingual ASR at 1600+ languages (5400+ zero-shot, ~10% CER on 78% of them) against Whisper’s 99.

From below: a single average WER hides which kind of audio a model handles. VibeVoice-ASR-BitNet publishes a 15-benchmark table where it beats Whisper on meeting and far-field speech by 5–11 points (AMI-sdm 25.87 vs 36.92) and loses to it on clean read speech (Libri-clean 2.41 vs 1.98) — with Parakeet winning the clean sets outright and losing every meeting set. Which model is “most accurate” depends entirely on which half of that table matches your audio, which is the same conclusion the roundup reaches by a different route.

The market shape

Mirrors the rest of speech-audio-ai: commercial APIs hold a thin accuracy edge (ElevenLabs Scribe ~3.3% EN, Deepgram 5.26% batch — stt-apis-comparison) over the open-source field (whisper, canary-qwen, Parakeet, Granite, Qwen3-ASR — open-source-stt-models), with the usual build-vs-buy crossover at high volume.

The defining 2026 trend — STT meets the LLM

Top accuracy now comes from SALM-style models that bolt an LLM decoder onto a speech encoder (canary-qwen = FastConformer + Qwen3; Granite-Speech; Qwen3-ASR) — recognition reframed as language modeling, the bridge to llm-providers-wiki (gemini, Qwen/Llama). whisper no longer leads WER but wins on ecosystem (MIT, languages, tooling).

The application layer — on-device dictation (2026-07)

The WER leaderboard is only half the STT story. The other half is what these engines get used for on a personal machine, where the axis flips from accuracy to privacy, offline operation, and desktop integration. vocalinux (Linux voice dictation) is the wiki’s first source on this layer: it wraps a chosen local backend — whisper-cpp, whisper, or vosk — and does the hard OS work of injecting text under both X11 and Wayland. It also surfaces the small/local pole the accuracy pages skip: vosk (Kaldi-based, ~50 MB, runs on a Pi) is picked for constraint, not WER, and whisper-cpp is what makes Whisper responsive on a laptop. So the open-weight STT wedge has a consumer end-point — assemble a private dictation tool from off-the-shelf engines, no cloud involved.

speech-audio-ai · text-to-speech · whisper · whisper-cpp · vosk · vocalinux · canary-qwen · open-source-stt-models · stt-apis-comparison · tts-benchmarks