Get Started Studio Demos Blog Research Talks Community

📚 Full list of publications

Here is a link to all of our research publications.

💾 Datasets

As part of the project, we open source some of the datasets that were used in our research.

🔎 Research highlights


DDSP

A library that lets you combine the interpretable structure of classical DSP elements (such as filters, oscillators, reverberation, etc.) with the expressivity of deep learning.

overview of DDSP

Papers

Blog Posts

Colab Notebooks

GANSynth

A method to synthesize high-fidelity audio with GANs.

Wave2Midi2Wave

A new process able to transcribe, compose, and synthesize audio waveforms with coherent musical structure on timescales spanning six orders of magnitude (~0.1 ms to ~100 s).

Music VAE

A hierarchical latent vector model for learning long-term structure in music

Onsets and Frames

We advance the state of the art in polyphonic piano music transcription by using a deep convolutional and recurrent neural network which is trained to jointly predict onsets and frames.

Latent Constraints

A method to condition generation without retraining the model, by post-hoc learning latent constraints, value functions that identify regions in latent space that generate outputs with desired attributes. We can conditionally sample from these regions with gradient-based optimization or amortized actor functions.

COCONET

An instance of orderlessNADE, Coconet uses deep convolutional neural networks to perform music inpaintings through Gibbs sampling.

Performance RNN

An LSTM-based recurrent neural network designed to model polyphonic music with expressive timing and dynamics.

Sketch RNN

A recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes.

overview of Sketch RNN

Papers

Blog Posts

NSynth

A powerful new WaveNet-style autoencoder model that conditions an autoregressive decoder on temporal codes learned from the raw audio waveform.

overview of NSynth

Papers

Blog Posts

Colab Notebooks