SheetSage2

Audio-to-lead-sheet transcription from multimodal-art-projection (m-a-p): a full song in, an editable ABC score with melody and chords out, plus MIDI parts (vocal melody, instrumental melody, chords), beats, downbeats, key and section structure as timed annotations. Read from the Hugging Face model card; arrived by Telegram 2026-09-16. A technical report is "forthcoming" — the card asks to cite the YuE paper (arXiv 2503.08638) and MERT instead, so there is no paper for this model yet. Corrected 2026-10-04 (claim drain): a paper now exists. YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality (arXiv 2609.33757, submitted 2026-09-27, Yuan, Pan, Jiang et al.) introduces SheetSage2 beside MERT2 and reports that it "leads 12 of 15 benchmark-metric pairs" in the authors' own lead-sheet transcription comparison. Those are the makers' numbers on a comparison they chose; nobody outside has re-run them.

Shape

• 57.2M parameters, F32 weights, BF16 inference — small, because the heavy lifting is the base: adapters over MERT-v2-FullSong, m-a-p's music-understanding encoder, merged at load time. Long songs are processed in overlapping windows.
• One call: model.transcribe("song.mp3", output_dir=...). Python 3.10–3.11, FFmpeg 6.1, PyTorch 2.8.
• CC BY-NC 4.0 on the weights — non-commercial only, which puts it in the same licence bucket as most of the music models on free-voice-clone-list.

The numbers, first-party

The card reports per-task scores on public MIR test sets: beat 89.01% F1 (GTZAN), downbeat 92.90% F1 (osu2017), chord 90.08% maj/min (osu2017), vocal melody 82.51% F1 (RWC-Pop), structure 80.51% accuracy (HarmonixSet). The card calls these "competitive or leading"; it lists comparators, but the comparison set is the authors' choice and there is no independent run. 6,264 downloads last month, 88 likes at read time.

What it adds here

The spoke's music corner has been about generation (elevenlabs's Eleven Music, the audio-gen architectures). This is the other direction — analysis of recorded music into symbolic form — and the first page here on it. Two things to keep: the whole system is an encoder plus small adapters, the pattern neural-audio-codec pages describe for speech applied to music understanding; and a melody F1 in the low 80s on pop vocals means roughly one note in five is still wrong, which is the gap between "editable lead sheet" and a lead sheet you would hand a band.

Runner-up spoke: music-tech (the output is a production artefact). Routed here because the object is an audio model with benchmark claims.

Tier T3 — first-party model card, no report, first-party comparison set.

Related

multimodal-art-projection · neural-audio-codec · free-voice-clone-list · elevenlabs