In this talk and live-demo, Taylor Berg-Kirkpatrick and Jacob Andreas present a new probabilistic model for transcribing piano music from audio to a symbolic form. Their model reflects the process by which discrete musical events give rise to acoustic signals that are then superimposed to produce the observed data. As a result, the inference procedure for their model naturally resolves the source separation problem introduced by the the piano’s polyphony. In order to adapt to the properties of a new instrument or acoustic environment being transcribed, we learn recording-specific spectral profiles and temporal envelopes in an unsupervised fashion. Their system outperforms the best published approaches on a standard piano transcription task, achieving a 10.6% relative gain in note onset F1 on real piano audio.

Taylor Berg-Kirkpatrick is a PhD candidate in computer science at the University of California, Berkeley. He works with professor Dan Klein on using machine learning to understand structured human data, including language but also sources like music, document images, and other complex artifacts. Taylor completed his undergraduate degree in mathematics and computer science at Berkeley as well, where he won the departmental Dorothea Klumpke Roberts Prize in mathematics. As a graduate student, Taylor has received both the Qualcomm Innovation Fellowship and the National Science Foundation Graduate Research Fellowship.

Jacob Andreas is a PhD student in computer science at UC Berkeley, advised by Dan Klein. His work focuses on model-based approaches to natural language semantics, using both formal representations of meaning and direct grounding in perception and action. Jacob received an MPhil from the University of Cambridge, which he attended as a Churchill Scholar, and a BS from Columbia University.