Advances in Automatic Piano and Music Transcription
Automatic Piano and Music Transcription
Many people, especially those who do not have much experience with transcribing music, think that transcription is a simple process of identifying notes and translating them into written notation. This is a dangerously simplistic view that ignores the complexities of musical sound and the broader purpose of transcription: it is a process of trying to understand what series of notational instructions would make a musician play the music that we hear, then figuring out how to translate those into a form that will let other musicians recreate that same sequence of sounds with their instruments.
The ability to automatically transcribe music into a notational form is a fascinating example of artificial intelligence, and it has a wide range of applications in the fields of music information retrieval (MIR) and human-computer interaction. However, the quality of the output from current automatic transcription systems is not yet good enough to be useful, as they contain large numbers of mistakes and unreadable or difficult-to-read notations.
Transcription is a complex task, and getting from a raw audio signal to a music score is a challenging endeavor even for human transcribers. There are many different subtasks involved in the complete music transcription process, including multipitch estimation, onset and offset detection, instrument recognition, beat and rhythm tracking, and interpretation of expressive timing and dynamics. The quality of these subtasks is highly dependent on the complexity and quality of the music being transcribed, which makes it very hard to produce high-quality results.
Advances in Automatic Piano and Music Transcription
Most recently, advanced machine learning techniques have been applied to the music transcription problem in order to improve its accuracy and robustness. For instance, sequence-to-sequence transfer learning has been successful in natural language processing, and it was also used in one of the recent studies on automatic piano transcription based on a transformer model [24].
Another approach to improve the performance of music transcription is to use frame-level analysis, which aims at modeling the time relations within data. It was implemented in the study on automatic piano transcription based on time delay neural networks [25]. Time delay networks are a type of network model which aims at representing temporal relations between consecutive frames. This allows for better onset/offset detection, as it will take into account the relation between two adjacent frames when detecting if there is a note in the middle of them.
The onset/offset detection is the first step to get from a raw audio signal to a MIDI file that contains the note onsets and their durations. MIDI is an international standard that defines the fundamental frequencies and their corresponding MIDI note names, along with their onset/offset times. The MIDI format is not an exact representation of a musical frequency and pitch, as there are several deviations from the true frequency value due to rounding and down-sampling in the MIDI standard.
The onset and offset timings are obtained by a frequency-to-MIDI conversion algorithm that uses a pre-defined frequency-MIDI conversion table. The onset and offset times are then passed to a note recognition method that uses an activation threshold to identify notes in the frequency-to-MIDI conversion table.