DiarizationLM-Gemma-4-E4B-v1
A 4B-parameter model Google posted to Hugging Face on 4 October 2026, reported by MIXED Reality News (Shane S. Ellison) the next day. It does one job: fix who-said-what in a transcript that already exists. It is a post-processing step, not a recogniser and not a diarizer. The model card says in bold that it is "not an officially supported Google product", so this is a code-and-weights release, not a service.
Tier T3. The source is a trade-press report of vendor-run numbers. The article quotes the model
card, and this page did not read the card itself (huggingface.co/google/DiarizationLM-Gemma-4-E4B-v1).
Every figure below is Google's, relayed. The paper behind the framework (DiarizationLM, Interspeech
2024) is cited in the article without a link and has not been read here.
What it does
The DiarizationLM framework sits after an ASR system and a diarization system. A prompt builder packs the words and their speaker labels into text, the LLM rewrites the labels, and a parser puts the result back on the transcript. Because it corrects an existing pipeline, results are reported relative to a baseline (a USM recogniser with turn-to-diarize), not as standalone accuracy. The metric is WDER, the share of words given to the wrong speaker.
Google credits the gains to its training method, Locality-Preserving Oracle Supervision, which targets turn boundaries and short backchannels ("mm-hm") while keeping a speaker's identity steady through a long monologue. Training was 10,000 steps with a rank-256 LoRA adapter on eight TPU v5p chips. The article does not say which datasets it trained on.
The numbers (WDER, vendor-reported)
| Set | Baseline | This model | Change |
|---|---|---|---|
| Fisher English (phone calls) | 5.32% | 2.99% | -2.33 pts |
| Callhome American English (phone calls) | 7.74% | 4.92% | -2.82 pts |
| ICSI (meetings) | 14.70% | 14.10% | -0.60 pts |
| AMI (meetings) | 15.68% | 14.89% | -0.79 pts |
All four are reported significant at p < 0.0001. Against the older DiarizationLM-8b-Fisher-v2 (Llama 3 8B), the 4B model scores 2.99% vs 3.28% on Fisher and 4.92% vs 6.66% on Callhome, with half the parameters. There is no 8B result for ICSI or AMI.
The split is the finding. The two telephone sets, two speakers on a clean channel, fall by about 40% relative. The two meeting sets, several speakers in a room, move by less than one point. The article's headline ("two barely") is fair. Significance shows the effect is real, not that it is large, and the meeting sets are small: ICSI is 3 meetings, AMI is 16.
Practical facts
- Licence: Apache-2.0, so not a restrictive-weights case; it can be paired with whisper or any model on open-asr-leaderboard.
- Size: merged 16-bit weights about 16.0 GB; two 4-bit GGUF builds at about 5.30 and 5.15 GB for llama.cpp and Ollama.
- Adoption at report time: 0 downloads, 9 likes (00:00 UTC, 5 October). A day-one reading, not a signal.
- Missing: no latency, no on-device figure, no WER or cpWER. Nobody can say yet whether a 5 GB language model per call is a sensible price for a 2-point gain on phone audio.
- Contrast in the same week: the article notes Nvidia's 100M-parameter diarizer (28 September), which labels speakers directly instead of repairing another system's output. Not read here.
Why it matters here
This is the first source in the spoke that is about diarization rather than reporting it as a side metric. It lands on the argument in synthesis that the speaker-labelling stage is where conversational transcription breaks: chime-8-dasr found DER does not predict the downstream word error, and chime-6 scored with cpWER because speaker mistakes count as word mistakes. WDER is a different metric again. It assumes the words are right and grades only the labels, which isolates the stage this model touches but says nothing about the end-to-end score. Two things follow. The reported gain is largest on the easy, two-speaker, near-field case and smallest on the multi-speaker meeting case that CHiME-style benchmarks stress, so the model should not be read as answering the far-field problem. And an LLM correcting labels from text alone cannot recover speakers the acoustic front end merged, which is the speaker-counting failure chime-8-dasr blames for most downstream errors. That second point is our inference from the two pages, not something the source says.
The backbone is gemma-4 (in ../llm-providers-wiki), so this is also a small instance of the
bridge the schema describes: an audio-pipeline stage riding on a text-LLM backbone, here fine-tuned
with LoRA rather than trained from scratch.
Related
speech-to-text · chime-8-dasr · chime-6 · on-device-streaming-asr-study