SenseVoice
A speech understanding foundation model from Alibaba’s speech team (the FunAudioLLM family), doing four tasks in one pass: ASR, spoken language identification, speech emotion recognition (seven categories) and audio event detection (BGM, applause, laughter, crying, sneezing, breathing, coughing). ASR covers Mandarin, Cantonese, English, Japanese and Korean. Code is MIT; the weights are not — they carry the FunASR Model Open Source License Agreement, which permits commercial use under stated conditions. ~9k★, 805 forks.
Only SenseVoiceSmall is released. A large variant appears in the benchmark tables and is not downloadable, so every number attributed to “SenseVoice-L” describes something the public cannot run.
The speed claim is architectural rather than a kernel trick: the model is non-autoregressive, so it does not decode token by token. The README puts SenseVoiceSmall at more than 5× Whisper-Small and 15× Whisper-Large at comparable parameter counts. CosyVoice is the generation-side sibling from the same group; SenseVoice listens, CosyVoice speaks.
T3, and note the repository owner. This copy sits under a QwenAudio GitHub organization while the model is FunAudioLLM/Alibaba work. Whether that org is the team’s own, a lab reorganization, or a mirror is not established from the page, and it matters for anyone deciding what to trust the weights against. The performance figures are first-party and self-run, with no independent WER on a common benchmark.
Where it sits in the spoke
The corpus’s STT entries are graded on transcription accuracy and how small they get: whisper as the ecosystem default, canary-qwen on WER, vosk and moonshine at the embedded end, vibevoice-asr-bitnet on quantized CPU throughput. SenseVoice is not competing on that axis. It returns who is speaking what language, in what emotional state, over what background sounds, and the transcript is one of four outputs.
That puts it between two branches this wiki has kept apart. speech-to-text treats audio as text to be recovered; audio-flamingo-3 and the audio-understanding (LALM) branch treat audio as something to reason about. SenseVoice does the second job without an LLM attached — fixed task heads, not open-ended reasoning — and it is fast because it refuses the autoregressive decoder that a reasoning model needs. So the branch this spoke opened with Audio Flamingo has a cheap, non-generative form, and the two are not substitutes.
Emotion recognition is the claim to hold at arm’s length. Seven discrete categories is a long-contested modeling choice in affective computing, and the README reports no per-category accuracy, no speaker demographics and no cross-corpus evaluation — the setting where SER results are known to collapse. The audio-event labels are the more defensible half of the same output.
Related
speech-to-text · whisper · canary-qwen · vibevoice-asr-bitnet · moonshine · audio-flamingo-3 · speech-audio-ai · open-source-stt-models