Get Started Studio Demos Blog Research Community

📚 Full list of publications

Here is a link to all of our research publications.

💾 Datasets

As part of the project, we open source some of the datasets that were used in our research.

🔎 Research highlights


A method to synthesize high-fidelity audio with GANs.

Music Transformer

A self-attention-based neural network that can generate music with long-term coherence.

overview of Music Transformer


Blog Posts


A new process able to transcribe, compose, and synthesize audio waveforms with coherent musical structure on timescales spanning six orders of magnitude (~0.1 ms to ~100 s).

Music VAE

A hierarchical latent vector model for learning long-term structure in music

Onsets and Frames

We advance the state of the art in polyphonic piano music transcription by using a deep convolutional and recurrent neural network which is trained to jointly predict onsets and frames.

Latent Constraints

A method to condition generation without retraining the model, by post-hoc learning latent constraints, value functions that identify regions in latent space that generate outputs with desired attributes. We can conditionally sample from these regions with gradient-based optimization or amortized actor functions.


An instance of orderlessNADE, Coconet uses deep convolutional neural networks to perform music inpaintings through Gibbs sampling.

Performance RNN

An LSTM-based recurrent neural network designed to model polyphonic music with expressive timing and dynamics.

Sketch RNN

A recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes.

overview of Sketch RNN


Blog Posts


A powerful new WaveNet-style autoencoder model that conditions an autoregressive decoder on temporal codes learned from the raw audio waveform.

overview of NSynth


Blog Posts

Colab Notebooks