How accurate is AI music transcription? We measured it

The short answer

Good enough for a first draft, not for a finished score. In our October 2, 2026 test on 12 public guitar recordings, Spotify’s open-source Basic Pitch reached an onset F1 of 0.744 and Mirelo’s Audio-to-MIDI 0.471, where 1.0 would mean every note found at the right pitch and time with nothing extra. Mirelo returned no notes for three of the six solo recordings, but it led on a 60-second band excerpt, 0.672 to 0.520. Note lengths were far less reliable than note starts, and none of these numbers says whether the written score is readable.

What does onset F1 measure?

Onset F1 scores a transcription note by note against a reference transcription, and it punishes missed notes and extra notes alike. A transcribed note counts as a match when its pitch is within 50 cents (a quarter tone) of a reference note and it starts within 50 milliseconds of it. Each reference note can be matched only once.

Two shares come out of the matching. Precision is the share of the model’s notes that match; recall is the share of reference notes the model found. F1 is their harmonic mean, so it stays low when either one is low. Say a passage has 100 notes. A model that writes 100 notes, 75 of them right, scores 0.75. A model that finds 90 of the 100 but writes 150 notes to do it has a recall of 0.9, a precision of 0.6 and an F1 of 0.72: the 60 extra notes cost it.

Onset+offset F1 adds a condition: each note must also end within 20% of the reference note’s length, or within 50 ms if that is longer. Spotify’s Basic Pitch paper uses onset F1 as its main measure because note ends are less objective; it names reverb, the sustain pedal and the way annotations are made. We report both. Neither one checks which instrument a note was given, the key signature or how the rhythm would be written.

How we tested

We used GuitarSet 1.1.0 (Xi et al., ISMIR 2018), a public dataset of 360 guitar excerpts of about 30 seconds: six guitarists play the same 30 lead sheets in five styles, first as an accompaniment (“comp”) and then as a solo over it. It was recorded with a hexaphonic pickup, which gives each string its own signal and let the authors largely automate the note annotations.

We took 12 full recordings, one solo and one comp per guitarist, using the mono microphone audio: 433 seconds in all. The choice was fixed before any model ran. For each guitarist and mode we took the file whose SHA-256 hash of “scorestarling-v1:” plus its name sorts first, after excluding three recordings with known annotation errors (GuitarSet issues #4 and #5: a duplicated note and wrong timings). The table of results by recording below names the 12 it picked.

Scoring used mir_eval 0.8.2 with its default tolerances, as described above, averaged per recording so that each one counts equally. Nothing was tuned: no timing offsets, no filtering, no alignment shift. Basic Pitch 0.4.0 ran on a Mac, and two full runs gave byte-identical output. Mirelo’s Audio-to-MIDI API, which reported its model as a2m-1.1, ran the same 12 recordings on October 2, 2026, without the optional instrument list its API has accepted since August 6, 2026, according to Mirelo’s changelog.

Four limits apply to everything below:

• Basic Pitch’s paper lists GuitarSet and Slakh among its training data, so both tests favor it. We can’t tell whether our 12 GuitarSet recordings were in its training split; the band excerpt comes from Slakh’s test split.
• Twelve guitar recordings and one 60-second band excerpt make a diagnostic sample, not a benchmark. Voice and recorded band performances are unmeasured, and piano is measured only on synthesized audio, with another model (see the questions at the end).
• Mirelo ran the guitar recordings once, on October 2. We haven’t retried the three where it returned nothing, for example by telling it to expect acoustic guitar. Sent again on October 5, the band excerpt came back with the same notes.
• None of these numbers rates the written score. A score-level measure such as MV2H, which also checks meter, voices and note values, is still to be done.

ScoreStarling uses Basic Pitch for one instrument or voice, and Mirelo, our band partner, for a band, so we are not neutral observers. That is why the method, the sample and its limits are spelled out here.

Basic Pitch vs Mirelo on guitar recordings

Basic Pitch came out ahead: far ahead on solo lines, narrowly on accompaniment.

Update, October 4, 2026: we ran the same 12 recordings again through ScoreStarling’s one-instrument option after two changes to it: a take steadily sharp or flat is retuned to A440 first, and a melody is written one note at a time. It scored 0.846 on the solos, 0.679 on the accompaniments and 0.763 overall, or 0.522 with note ends. The table keeps the October 2 results of Basic Pitch on its own.

Mirelo returned no notes at all for three of the six solo recordings. They count as zero in its solo average, because an empty result is what a user would have received, and we have not rerun them. Where it did return notes, the picture was mixed. On the first recording, 00_BN3-154-E_solo, the two engines were level on note starts (0.809 for Basic Pitch, 0.814 for Mirelo), but Mirelo’s note ends were further off (onset+offset F1 0.320 against 0.524), and it put the 154 BPM piece at about 76.9 BPM, half the tempo. On accompaniment, Mirelo was clearly better on two of the six recordings. Across all 12, its onset+offset F1 was 0.224.

Basic Pitch’s 0.744 sits close to what its authors report on their own 72-recording GuitarSet test split: 0.79 onset F1, and 0.56 with note ends. The engines also differed in speed. Basic Pitch got through the 433 seconds of audio in 54 seconds on a Mac, which is not a production timing; on the first recording it took 6.1 seconds and Mirelo 37.

What happens with a full band?

A mix is harder, and there Mirelo led. We took the first 60 seconds of Track01881 from the test split of Slakh2100, a dataset of 2,100 songs rendered from MIDI with sample-based virtual instruments, so the reference notes are exact. The excerpt has steel-string and jazz guitar, piano, bass, vibraphone and drums, all playing within that minute. Pooled over every pitched instrument, ignoring which instrument a note was assigned to, onset F1 was 0.520 for Basic Pitch and 0.672 for Mirelo, scored on the MIDI file each engine returns.

Basic Pitch writes one combined part and no drums. Mirelo writes a part per instrument, so only Mirelo can be scored by instrument, grouped here into General MIDI instrument classes:

Mirelo’s onset F1 by instrument one 60-second band excerpt, 0 to 1

Piano0.827

Drums0.698

Guitars0.681

Bass0.455

Vibraphone0.137

Telling the two guitars apart was harder. Scored part by part from Mirelo’s note list on October 2, the steel-string guitar reached 0.364 and the jazz guitar 0.148; the MIDI file starts its notes about 20 ms earlier than that list, so its figures differ slightly. Mirelo also reported 92 notes for instruments that are not in the song: organ, synth pad, violin and cello. Drum strokes are scored on timing alone, whichever drum. This is one song, rendered rather than recorded, so read it as a direction rather than a ranking.

Which notes are most likely to be wrong?

With Basic Pitch, the ones it wrote with a low velocity. Basic Pitch sets each note’s MIDI velocity to 127 times the model’s mean activation over that note, so velocity doubles as a confidence score. Across the 2,715 notes it wrote for the 12 guitar recordings, matched with the same 50 ms window, low-velocity notes were mostly wrong:

Notes Basic Pitch got right, by velocity 2,715 notes from 12 guitar recordings

Below 5021%

50–5940%

60–7967%

80–12786%

Velocity separates right from wrong notes with an AUC of 0.77, where 0.5 is a coin toss and 1.0 is perfect; note length manages 0.53, so short notes are not a useful warning sign. Deleting every note below velocity 50 would raise the overall onset F1 from 0.744 to 0.765, but one of those notes in five is correct. ScoreStarling’s score review therefore lists them as doubtful rather than deleting them, and leaves the decision to your ears. If you run Basic Pitch yourself, coloring or sorting notes by velocity in a piano roll gives you the same shortlist.

Why don’t published accuracy figures agree?

They measure different things on different recordings, and some don’t say what they measured.

Figures can be compared only when the metric, the tolerance and the recordings match. Our Basic Pitch result can sit beside its paper’s, since both use onset F1 on GuitarSet, though on different subsets; it can’t sit beside MuScriptor’s. The ScoreCloud page doesn’t say how pitch accuracy was counted, on how many recordings, or whether timing mattered. MuScriptor is the open research model that Mirelo’s Audio-to-MIDI Pro grew out of, and Mirelo describes its production model as more accurate. Mirelo’s post of September 17, 2026 says an updated model improved note and instrument accuracy “in our evaluations”, without publishing figures.

Do accurate notes make a readable score?

No. Onset F1 compares notes in seconds; it says nothing about whether the written score reads well. A score also needs a key signature, sensible sharps and flats, written rests, voices and a meter, and plain MIDI from a model like Basic Pitch carries none of them. On the same Basic Pitch notes, our first notation writer produced scores with no key signature and no visible rests in all 12 cases, while MuseScore Studio’s MIDI importer wrote the annotated key in 10 of 12 and showed every rest. Mirelo’s own MusicXML had the annotated key in 5 of its 9 completed cases, always on a single treble staff.

A transcription can score well and still be hard to read, and a tidy page can still hold wrong notes. The full before-and-after is in Why MIDI imports look messy in MuseScore.

Which transcription engine should you use?

Choose by the recording. Our reading of these results:

• One instrument or one melody line: Basic Pitch. It was far ahead on solo guitar, it is free and open source, and its README says it works best on one instrument at a time.
• A band, or a mix where you need separate parts or drums: a multi-instrument model such as Mirelo’s, since Basic Pitch writes a single combined part.
• Guitar accompaniment: either. They were close on average and each did better on different recordings, so try both on a short excerpt if the choice matters.
• Voice: we haven’t measured it yet. Piano: we have measured only our own solo-piano option, on synthesized audio, so we can’t rank engines for it.

Whichever engine you use, check rhythm and note lengths by ear. The measurements above, from October 2 and the October 4 re-run, leave out our solo-piano option. ScoreStarling uses Basic Pitch for one instrument or voice and a piano model for solo piano, both free, and Mirelo for a band, which uses credits (checked October 7, 2026). Each covers the first five minutes of an upload of up to 100 MiB. A voice or one instrument starts in 4/4; solo piano works out its meter, and a band score’s meter is found from the playing or chosen before it starts. Guitar, bass and ukulele parts can also be read as TAB, without bends, slides or hammer-ons; see the specifications.

Sources

1. GuitarSet 1.1.0 — Xi, Bittner, Pauwels, Ye and Bello, Zenodo (CC BY 4.0)
2. GuitarSet project page — the GuitarSet authors
3. GuitarSet issues — marl/GuitarSet, GitHub
4. Slakh2100 — Manilow, Wichern, Seetharaman and Le Roux, Zenodo (CC BY 4.0)
5. Cutting Music Source Separation Some Slakh — Manilow et al., WASPAA 2019
6. A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimation — Bittner et al., Spotify, 2022
7. Basic Pitch and its note_creation.py — Spotify, GitHub
8. mir_eval.transcription — mir_eval documentation
9. MuScriptor: An Open Model for Multi-Instrument Music Transcription — Rouard et al., 2026
10. Audio-to-MIDI Pro: full mixes to editable MIDI — Mirelo, September 17, 2026
11. Audio to MIDI converter and Changelog — Mirelo
12. How Accurate Is Automatic Music Transcription? — ScoreCloud

Questions and answers

Can AI transcribe music from an MP3?

Yes, into MIDI note by note, with errors to fix. Models such as Spotify’s Basic Pitch work best on one instrument; full mixes need a multi-instrument model and come out less accurate. In our October 2, 2026 test the best group result was an onset F1 of 0.818, Basic Pitch on solo guitar (0.846 in an October 4 re-run, after two changes to how ScoreStarling uses it), so plan to correct some notes by ear.

Is Basic Pitch accurate enough to make sheet music?

For a first draft from one instrument, yes. On 12 guitar recordings it scored 0.744 onset F1 for note starts but 0.508 once note ends counted, and its MIDI has no key signature, rests or voices, so a notation step has to add them. Why MIDI imports look messy in MuseScore shows what that step changes.

How can I measure transcription accuracy myself?

You need a reference transcription you trust, with note times in seconds. With one, the open-source mir_eval library scores note matches with the tolerances we used: 50 ms for the start, 50 cents for pitch and, for note ends, 20% of the note’s length or 50 ms. Without one, loop short passages and compare the score with the recording by ear.

Why did Mirelo return no notes for some guitar solos?

We don’t know. Three of the six solo recordings came back empty on October 2, 2026. We ran them without Mirelo’s optional instrument list and haven’t retried them. On accompaniment it came close to Basic Pitch, and it led on the multi-instrument excerpt.

How accurate is AI transcription for singing or piano?

For singing, we can’t give you our own numbers yet. For piano, only on synthesized audio: on the opening bars of 25 pieces, each played twice with rubato (50 performances, measured October 6, 2026), our solo-piano option, built on Kong and colleagues’ piano model, heard about 92% of the written notes but got the beat right in only 35 of the 50, and the beat, meter and pickup in 26. Basic Pitch’s authors built it to work across instruments, voice included. A clear, dry recording of one voice or instrument gives any model its best chance.