Skip to main content

GPT-3 MIDI sequence autocompletion toy example

Comments

GPT-3 MIDI sequence autocomplete

This is a tiny example created from playing around with GPT-3 at the Deep Learning Labs hackathon in Stockholm, the 20th of May, 2022.

Naively encoding MIDI events to a text representation, to autogenerate melodies from a prompt melody, doesn't feel like it should work at all to me. However, while this type of MIDI-to-text conversion is clearly limited, it's interesting to note that the predicted output sequence of tokens is both 1) formatted consistently enough that it can be encoded back to MIDI events, and 2) that the subsequent melody actually sounds fairly musical and almost like an actual continuation of the prompt melody.

-- Carl Thomé

In [1]:
%load_ext dotenv
%dotenv
In [2]:
import pretty_midi

midifile = pretty_midi.PrettyMIDI("song.mid")

# TODO Sum up all instruments instead of taking the lead instrument.
midifile.instruments = [midifile.instruments[7]]
In [3]:
import IPython.display as ipd


def sonify(midifile: pretty_midi.PrettyMIDI, start: float, samplerate: int):
    waveform = midifile.synthesize(fs=samplerate)[int(start * samplerate):]
    audio = ipd.Audio(waveform, rate=samplerate)
    return audio


# Listen to original input MIDI file.
sonify(midifile, start=0.0, samplerate=44100)
Out[3]: