GPT-3 MIDI sequence autocompletion toy example

2022-05-21 16:56

Carl Thomé

GPT-3 MIDI sequence autocomplete¶

This is a tiny example created from playing around with GPT-3 at the Deep Learning Labs hackathon in Stockholm, the 20th of May, 2022.

Naively encoding MIDI events to a text representation, to autogenerate melodies from a prompt melody, doesn't feel like it should work at all to me. However, while this type of MIDI-to-text conversion is clearly limited, it's interesting to note that the predicted output sequence of tokens is both 1) formatted consistently enough that it can be encoded back to MIDI events, and 2) that the subsequent melody actually sounds fairly musical and almost like an actual continuation of the prompt melody.

-- Carl Thomé

In [1]:

%load_ext dotenv
%dotenv

In [2]:

import pretty_midi

midifile = pretty_midi.PrettyMIDI("song.mid")

# TODO Sum up all instruments instead of taking the lead instrument.
midifile.instruments = [midifile.instruments[7]]

In [3]:

import IPython.display as ipd


def sonify(midifile: pretty_midi.PrettyMIDI, start: float, samplerate: int):
    waveform = midifile.synthesize(fs=samplerate)[int(start * samplerate):]
    audio = ipd.Audio(waveform, rate=samplerate)
    return audio


# Listen to original input MIDI file.
sonify(midifile, start=0.0, samplerate=44100)

Out[3]:

GPT-3 MIDI sequence autocomplete¶

Comments