Piano Genie
Abstract
We present Piano Genie, a generative musical instrument
which allows non-musicians to play the piano. With Piano
Genie, a user performs on a simple interface with eight buttons,
and their performance is decoded into the space of
plausible piano music in real time. To learn a suitable mapping
procedure for this problem, we train recurrent neural
network autoencoders with discrete bottlenecks: an encoder
learns an appropriate sequence of buttons corresponding
to a piano piece, and a decoder learns to map
this sequence back to the original piece. During performance,
we substitute a user’s input for the encoder output,
and play the decoder’s prediction each time the user
presses a button. To improve the interpretability of Piano
Genie’s performance mechanics, we impose musically-informed
constraints over the encoder’s outputs.
which allows non-musicians to play the piano. With Piano
Genie, a user performs on a simple interface with eight buttons,
and their performance is decoded into the space of
plausible piano music in real time. To learn a suitable mapping
procedure for this problem, we train recurrent neural
network autoencoders with discrete bottlenecks: an encoder
learns an appropriate sequence of buttons corresponding
to a piano piece, and a decoder learns to map
this sequence back to the original piece. During performance,
we substitute a user’s input for the encoder output,
and play the decoder’s prediction each time the user
presses a button. To improve the interpretability of Piano
Genie’s performance mechanics, we impose musically-informed
constraints over the encoder’s outputs.
Research Areas
