How to fix an AI music transcription

What goes wrong in an automatic transcription?

A transcription model hears pitches and times; a score also needs a key, a meter, bar lines and note values someone can read. Five errors account for most of what we see in drafts:

• Ghost notes. A piano string sounds overtones with each note, and the model can hear the octave above as a key of its own, struck at the same moment.
• Pieces of one note. A held note can come back as two, a singer’s slide into a note as a short wrong note before it, a vibrato as a wobble between two pitches.
• The wrong key. Every pitch can be right and the signature wrong: Für Elise’s A minor needs no sharps, but its G sharps and D sharps make a model guess E major.
• The wrong meter. Note values doubled or halved (6/8 for 3/8), or a 3/4 piece read in 4/4.
• Bar lines in the wrong place. A piece that starts on an upbeat needs a short first bar, and a steady recording drifts off its bar lines when the tempo is a fraction of a beat a minute out.

How can you check a transcription yourself?

Play the score against the recording, a few bars at a time, and look for these:

1. Notes stacked an octave apart that the recording does not play: the upper one is usually a ghost.
2. Where phrases end. The note they come to rest on, and its chord, is usually the key; compare it with the key signature.
3. Count the first phrase. If its strong beats fall in the middle of bars, the meter or the first bar line is wrong.
4. Very short notes before longer ones a step away: in singing these are slides into the note, not notes of the tune.
5. Rests and ties scattered through a passage played legato: the notes are probably right and their lengths written badly.

What does an AI proofreading pass do?

ScoreStarling’s AI refine is a switch in the upload form for solo piano and for one instrument or voice. After the transcription, a language model reads the draft and edits it with the tools a person has in the editor, in a fixed order. We took that order from the strongest model we tried, which proofread six drafts with the same tools and no limit on time or steps:

It is told the recording is the judge: never re-transcribe, arrange, simplify or transpose. Its last message says what it changed and which bars to listen to again.

What does a fixed transcription look like?

Here is the opening of the Für Elise test, as the draft wrote it and after the pass. The draft is hard to read but holds every played note; the pass deleted the copies, set Beethoven’s key and meter, and gave each hand its own staff.

The 19-second Gymnopédie recording on our piano recording to sheet music page went wrong another way: the draft is in 4/4, with the left hand’s chords on the upper staff among the melody. Both runs of the pass ended with Satie’s notes in all eight bars, every pitch, note value and staff, in 3/4 and D major.

How well does it work?

We ran it twice on each recording with a reference transcription, on October 12, 2026, with the model it uses, OpenAI’s GPT-6.1 Sol. A note counts as right when a reference note has its pitch and starts within 0.1 seconds; F1 combines missed and extra notes, so 1.0 is every note and nothing else.

On two sung recordings from a research dataset whose licence keeps them off this site, F1 went from 0.78 to 0.89 and from 0.79 to 0.87, and the share of bars that start with a note rose from about a third to nine in ten or more.

Seven recordings, six of them under a minute, make a diagnostic sample, not a benchmark. The guitar recordings gain little because their drafts’ errors are mostly missed or merged notes, which a proofreading pass cannot recover.

What can’t it do?

• Bring back a note the draft missed. Every gain in the table came from removing extra notes: missed notes stayed missed in every test.
• Tell every real note from an extra one. On four of the five recordings with a reference it removed one to six real notes along with the extra ones; on the accompaniment, six of 117.
• Judge a recording without a reference for you. On a 2:48 pop piano recording it removed the draft’s ghost octaves and kept the left hand’s figure on the lower staff, but whether every removed note was a ghost can only be checked by ear.
• Work on band recordings, which have several parts in one mix.

How long does it take?

A few minutes after the transcription finishes: one to four minutes on our test recordings of 19 to 82 seconds, and under three minutes on the 2:48 piano recording. The score is ready when the pass ends, with a few sentences on what it changed.