Voice cloning

Synthesizing speech in a target speaker’s voice from a short reference sample — ideally zero-shot (no per-voice fine-tuning). A major capability axis that splits the text-to-speech field.

Who has it (and who doesn’t)

Has zero-shot cloning: orpheus, Chatterbox (strong), CosyVoice 2, IndexTTS-2 (which also separates timbre from emotion for dubbing), fish-audio-s2-pro open-source-tts-models.
No cloning by design: kokoro — it ships fixed preset voices, trading the capability for a tiny 82M footprint tts-models-2026-benchmark.

So “can it clone a voice?” is a clean dividing line: presence of cloning often correlates with larger models and conditioning on reference audio, while the smallest efficiency models drop it.

The dividing line stopped dividing (2026-08-03)

A 36-model census of open TTS (free-voice-clone-list, April 2026) marks 35 of 36 with zero-shot cloning, the sole exception being supertonic-2 at the small, fast end. The list sorts on languages, streaming and licence instead, which is what a capability looks like once it becomes table stakes.

That census also credited kokoro with cloning, contradicting the two roundups this page was built from. Checked and settled the same day against the Hugging Face model card: Kokoro-82M does not support cloning, and its voices/ directory holds 54 .pt voicepack tensors selected by file, with no reference-audio path. So kokoro remains this page’s example of a model that drops the capability by design, now on primary evidence rather than on two secondary sources agreeing.

The size correlation is also weakening. kittentts claims cloning at 15M parameters, a fifth of kokoro‘s footprint, which if it holds cuts the link between cloning and model size that this page was written around. Unverified — the only source is a curated list with no measurements.

Reference-sample length has converged too: entries across the catalogue advertise cloning from a three-to-ten-second sample, matching the three seconds luxtts claims. Whatever cloning cost in data, it no longer costs much.

A grounded exemplar: XTTS

The open zero-shot cloner with a peer-reviewed primary is XTTS (Casanova et al., Coqui, INTERSPEECH 2024 — xtts-paper). It pins down two things the roundups only implied. First, cloning travels across languages: XTTS “was trained in 16 languages and achieved state-of-the-art results in most of them” — one zero-shot model cloning a voice across many tongues, not a per-language system xtts-paper. Second, it locates cloning in a second architectural lineage: XTTS “builds upon the Tortoise model,” i.e. the autoregressive route to cloning, distinct from the codec-token route (neural-audio-codec) — two paths to the same capability. And it is openly released, which is what makes zero-shot cloning an off-the-shelf commodity (the permissive side of open-weight-tts‘s license split) and gives the consent problem below its teeth.

Adjacent capabilities

Cloning sits alongside emotion/style control (guided emotion in orpheus; Hume Octave 2 inferring emotion from meaning) and multi-speaker dialogue (Dia, sesame-csm). Together these “controllability” features, not just raw naturalness, increasingly differentiate models.

Note

Voice cloning carries obvious consent/impersonation risk; the wiki tracks it as a technical capability, but it is the capability most entangled with abuse and licensing/consent questions. The catalogues that distribute the capability say nothing about it — see audio-deepfake for the census.

Cloning a voice vs. editing a recording (added 2026-08-06)

voicecraft does something this page’s axis does not cover. Cloning conditions a new utterance on a reference voice; VoiceCraft’s first task is infilling an existing recording — replacing words inside real audio while the surrounding speaker, prosody and room stay put — with zero-shot TTS falling out as the case where everything is infilled. So “can it clone a voice?” has a sharper sibling: can it alter a recording that already exists? The census above says cloning is table stakes; editing is not, and no leaderboard in this wiki ranks it.

text-to-speech · xtts-paper · orpheus · sesame-csm · fish-audio-s2-pro · kokoro · kittentts · supertonic-2 · free-voice-clone-list · open-source-tts-models · audio-deepfake · voicecraft · open-weight-tts