Get Started Demos Blog Research Community

Magenta Studio (Ableton Live Plugin)


This page is for the Ableton Live integration of Magenta Studio. If you're looking for the standalone version instead, click here.

Table of Contents

Overview

Magenta Studio is a MIDI plugin for Ableton Live. It contains 4 tools: Continue, Generate, Interpolate, and GrooVAE, which let you use the Magenta models on your MIDI clips from the Session View.

side by side of 4 plugins titled continue, interpolate, groovae and generate 4 bars

Installation

Requirements

Installation

The version of Max/MSP currently bundled with Ableton Live Suite (as of 10.0.5) does not support node.script, which is used in this application. To get around this, you can point Ableton Live to your local copy of Max/MSP by following these steps:

  1. Go to your Live Preferences. In the File/Folder tab, there is a section titled "Max Application".
  2. Click "Browse".
    Live preferences section titled Max Application with Browse button highlighted
  3. Navigate to your local copy of Max/MSP (version 8 or higher). When it's been selected, your preferences will look something like this:
    Live preferences section titled Max Application with /Application/Max.app highlighted
  4. Unzip the downloaded file.
  5. Within the folder, there is a file called magenta.amxd. Drag magenta.amxd into any available MIDI track within Live. Don't move that file out of the container folder, as the relative file path's must be maintained for it to work.
    Animation of magenta.amxd dragged from folder into Live track

Usage

Launching the plugin

Each of the tools can be launched by clicking on its name within the plugin, which will launch a separate window.

Clip selection

All of the plugins work by choosing one or more clips from Ableton’s Session View. You must choose a track before selecting your clip. Only MIDI tracks will show up as options. Once all of your selections are made, the Generate button will become enabled.

Temperature

All of the plugins have a temperature slider. Temperature is a parameter used for sampling in the last layer of the neural network. You can think of it as controlling randomness: higher values produce more variation and sometimes even chaos, while lower values are more conservative in their predictions.

Limitations

Melody input is limited to monophonic melodies (one note at a time), and drums input uses this MIDI mapping. Notes outside this range will be mapped to these 9 instruments:

Instrument Pitch
Bass drum/Snare drum 36/38
Closed/Open hi-hat 42/46
Low/Mid/High tom 45/48/50
Crash/Ride cymbal 49/51

Continue

Continue uses the predictive power of recurrent neural networks (RNN) to generate notes that are likely to follow your drum beat or melody. Give it an input clip and it can extend it by up to 32 measures. This can be helpful for adding variation to a drum beat or creating new material for a melodic track. It typically picks up on things like durations, key signatures and timing. It can be used to produce more random outputs by increasing the temperature.

How to use

Select a clip which you would like to extend, then click Generate. The output clips will be added to the clip slots after your selected clip.

Generate

Generate is similar to Continue, but it generates a 4 bar phrase with no input necessary. Choose where you would like the output to go, the number of variations, temperature, and click Generate. This can be helpful for breaking a creative block or as a source of inspiration for an original sample.

Under the hood, Generate uses a Variational Autoencoder (VAE) that has been trained on millions of melodies and rhythms to learn a summarized representation of musical qualities. Generate chooses a random combination of these summarized qualities, and decodes it back to MIDI to produce a new musical clip.

How to use

Generate does not require any input files, so the clip selection determines where you’d like the output clips to go.

Interpolate

Unlike the other plugins, Interpolate takes two drum beats or two melodies as inputs. It then generates up to 16 clips which combine the qualities of these two clips. It’s useful for merging musical ideas, or creating a smooth morphing between them.

Interpolate also uses a Variational Autoencoder (VAE) similar to Generate. One way to think of the VAE is as a mapping from MIDI to a compressed space in which similar musical patterns are clustered together. Each of your input patterns is represented by a position on this map. Interpolate draws a line between these positions and returns clips along this line. The number of returned clips is set by the “steps” slider.

How to use

Interpolate requires two inputs, and these inputs must be on the same track. The outputs are inserted after the second clip. The clips should be the same length and less than 4 measures.

GrooVAE

GrooVAE adjusts the timing and velocity of an input drum clip to produce the “feel” of a drummer’s performance. This is similar to what a “humanize” plugin does, but achieved in a totally different way.

We recorded 15 hours of real drummers performing on MIDI drum kits. These recordings were quantized, removing all velocity and microtiming and were used to train a neural network to predict the unquantized beats as the output. This is similar to the method used in Google’s automatic language translation, but instead of words we are translating from quantized drum patterns to human performances.

How to use

GrooVAE takes one clip as an input and places the output clip one slot below the input.