BlogStory

An AI that transcribes music like a musician: one real run

For pianists and singers who want a score they can trust from a recording: how Agent Transcription listens, looks the piece up, writes against the recording and proofreads, and where it still needs you.

Written by
Published
Updated
Length
9 min read

The short answer

An AI transcription agent works on a recording the way a careful musician does: it listens, looks up the original and published scores, writes the piece out against the recording, then proofreads its own drafts and says what it isn’t sure of. A one-pass converter does only the first step. On a 19-second recording of Satie’s Gymnopédie No. 1, ScoreStarling’s Agent Transcription read 3 sources, checked 4 drafts and fixed 28 notes in about 6 minutes.

What does an AI transcription agent do that a converter doesn’t?

A converter runs a transcription model over the audio once and writes out what it heard. An agent treats that result as a first draft and keeps working on it. Audio-to-score converters, including the quick draft every ScoreStarling score gets, predict notes from the sound alone: they don’t know which piece it is, whether it was published, or how its composer wrote it down. An agent is a language model that can use tools in a loop until the job is done. Ours, Starling, can open web pages, PDFs and MusicXML files, read the algorithm’s draft, write new versions of the score and look at their engraved pages.

  1. ListenA first draft from the recording, to check
  2. Look it upThe original, published scores and arrangements
  3. WriteThe whole piece against the recording, bar by bar
  4. ProofreadEach draft read over, what’s wrong fixed, doubts noted
The four steps Agent Transcription takes. Writing and proofreading repeat until a draft needs no more changes.
The quick draft and Agent Transcription, as of November 8, 2026
What it doesQuick draftAgent Transcription
Listens to the recordingYesYes
Looks up the original and published scoresNoYes
Follows the original when it finds itNoYes
Checks its own scoreNoYes
Says what it isn’t sure ofNoYes
TimeA minute or twoUsually 10 to 20 minutes

What happened on a real recording?

We ran it on November 7, 2026, on the first eight bars of Gymnopédie No. 1 played by Robin Alciatore for Musopen: 19 seconds of solo piano, the clip our piano recording to sheet music page plays. Every step below comes from the run’s own log of what it opened and checked, timed from the start.

  1. Opened satie-gymnopedie-1.mxl, a MusicXML file of the piece on noctavepiano.com.

  2. Opened gymnopedie_1-a4.pdf, the Mutopia Project’s edition.

  3. The algorithm’s draft of the recording was ready to check.

  4. Opened the noctavepiano.com page for the piece.

  5. Checked its first draft, then three more at 4:02, 4:39 and 5:18.

  6. Delivered the score.

One run, minutes and seconds from its start. It began looking the piece up before the algorithm’s draft was ready, so the two waits overlapped.

While it works, the workspace shows a small studio that acts out each of those steps, with every source it opened and every draft it checked underneath. Each source links to the original page.

A pixel-art study with Starling at a desk saying Starling’s score is ready; below, three sources and four drafts as thumbnails
The end of the same run in the studio: 3 sources and 4 drafts, each one shown as it was opened or checked.

What did it change?

Every pitch in the algorithm’s draft was right; 28 notes were written wrong. Five notes the draft had put on the lower staff moved up to the right hand’s staff, and 23 notes had the wrong length. Starling also changed the tempo and dynamics marks in bars 1, 5 and 7, writing in Satie’s Lent et douloureux, the pp and the hairpins.

What changed between the algorithm’s draft and Starling’s score, 28 notes in all. The same numbers are in the table below.
Notes Starling changed in the algorithm’s draft of the 19-second recording
Kind of changeNotes
Note lengths changed23
Moved to the right hand5
Pitches changed0

The reason is what a recording can and can’t tell you. It tells you which keys go down and when. It doesn’t say how long the composer wrote each note or which hand plays it, and in this piece the sustain pedal holds the sound long after the keys are up. The published score settles both. Satie writes a dotted half note in the bass and, after a quarter rest, a half-note chord in each bar, and Starling’s version follows that.

The algorithm’s draft of the eight bars, with the notes Starling found written wrong in red
The algorithm’s draft, with the notes Starling found written wrong in red.
Starling’s version of the eight bars in 3/4, with the changed notes and marks in blue and its note that pedal changes aren’t marked
Starling’s version, with what it changed in blue. On the Agent Transcription page you can play the recording, the algorithm’s draft and Starling’s version.

Why does it look up the published score first?

Because for a piece that has been published, the score already answers what a recording can’t: note lengths, which hand plays what, and the composer’s own marks. Starling follows a score only when it is the version the recording plays. An arrangement, a simplified edition or a cover in another key would be a different piece of writing, so it then writes from the recording instead. It uses the recording to line every bar up, reads only what is openly available, and doesn’t get around paywalls.

In this run, both scores it used were free. The noctavepiano.com files are marked CC0 1.0 (public domain) and were converted, with no note changed, from the Mutopia Project’s edition, which Evin Robertson typed from the Dover edition and placed in the public domain. Satie wrote the piece in 1888 and died in 1925.

For your own song, an improvisation or an arrangement nobody has published, there is nothing to look up. Starling then writes from the recording and the algorithm’s draft, and the proofreading matters more.

What did it say it wasn’t sure of?

Two things, which it wrote beside the score in its own words:

Pedal changes and individual key releases aren’t marked; note lengths follow the published score

Playback follows the recording’s pace bar by bar; small differences in when notes start are approximated

Both are fair. The pianist’s pedaling is audible but not written in this score, so if you want to play it the way the recording sounds, listen for where the harmony is cleared. A list like this tells you which bars to check by ear first, which is the part of proofreading people most often skip.

How can you do the same by hand?

You can follow the same steps without any AI. They take longer, but they are how careful transcribers have always worked:

  1. Name the piece and the version. Title, composer and, for a cover or an arrangement, the arranger. Check the key the recording is in, because performers often transpose.
  2. Look for a published score. For public domain works, try IMSLP (its Gymnopédies page lists Satie’s 1888 manuscript and his own first edition) and the Mutopia Project, which typesets free editions of public domain music. For anything under copyright, buy it from the publisher or a licensed store.
  3. Check the edition against the recording. Same key, same repeats, and the same notes in a few bars you can hear clearly. If they differ, the recording is a different version, and you should transcribe it rather than copy the edition.
  4. Mark where each bar starts in the recording, with markers in an audio editor or a slow-down app, so you compare the same bars.
  5. Proofread bar by bar while listening, and write down what the recording can’t settle, such as pedaling, voicing or rubato, instead of guessing.

If you are writing a piece out from scratch, how to transcribe music by ear covers the listening, and how to fix an AI music transcription covers the errors an algorithm’s draft usually has.

How long does it take, and what does it cost?

A whole piece usually takes 10 to 20 minutes; the 19-second test took about 6. The first whole-piece runs for users after the launch on November 7, 2026 took 8 to 17 minutes. You can close the page: it keeps working and emails you when the score is ready, and the algorithm’s quick draft is there to read in the meantime.

With Plus or purchased credits, the opening is free to preview. The whole piece is 150 credits for up to 2 minutes of music, then 25 credits for each minute started after that, up to 6 minutes, and you decide after you have seen the opening. At top-up prices 150 credits is about US$3.75 (checked November 8, 2026). It is in beta. You get an editable score, every download format and no watermark.

What can’t it do yet?

  • This article shows one run on one clean solo piano recording. It is an example, not a benchmark: here the algorithm’s draft already had every pitch right, and a busier recording leaves the agent more to fix and more to miss.
  • Solo piano works best. One instrument or one voice works too; full bands aren’t supported yet.
  • Up to 6 minutes of music.
  • It doesn’t always get everything right. It tells you what it isn’t sure of, and you can change any note in the editor or ask Starling to.
  • It follows a published score only when it is openly available. It never gets around a paywall, so for a piece under copyright it writes from the recording.

Its method is public: Agent Transcription follows the transcription skills in ScoreStarling’s public plugin, which describe how to research a piece, write it out against the recording and proofread it.

Sources

  1. Erik Satie, Gymnopédie No. 1, played by Robin Alciatore for Musopen, released into the public domain: Wikimedia Commons. Our clip is its first eight bars.
  2. Noctave, Gymnopédie No. 1 (Satie): free piano sheet music, PDF and MusicXML under CC0 1.0, converted from the Mutopia Project edition by Evin Robertson after the Dover edition. Read November 8, 2026.
  3. IMSLP, 3 Gymnopédies (Satie, Erik): the 1888 manuscript and the composer’s own first edition. Read November 8, 2026.
  4. ScoreStarling, Agent Transcription run log of November 7, 2026 (sources opened, drafts checked, delivery), shown on the Agent Transcription page.

Questions and answers

Is there an AI that looks up the original sheet music when it transcribes?

Yes. ScoreStarling’s Agent Transcription looks up the original and published scores before it writes. When it finds the score of the version your recording plays, it follows that score and uses the recording to line it up. When there is none, it writes from the recording.

Does it copy sheet music that is under copyright?

It reads only what is openly available and doesn’t get around paywalls. In our test run, both scores it used were free, public domain files of a piece by Erik Satie, who died in 1925.

Why can an AI hear every note right and still write the score wrong?

A recording tells you which keys go down and when. It doesn’t say how long the composer wrote each note or which hand plays it, and the sustain pedal blurs lengths. In our run, the algorithm’s draft had every pitch right, but 28 notes were on the wrong staff or had the wrong length.

How long does Agent Transcription take?

Usually 10 to 20 minutes for a whole piece. You can close the page; it keeps working and emails you when the score is ready. A quick draft made by an algorithm is ready in a minute or two in the meantime.

Bring any music Leave with a score