Song From PI

A Musically Plausible Network for Pop Music Generation


We present a novel framework for generating pop music. Our model is a hierarchical Recurrent Neural Network, where the layers and the structure of the hierarchy encode our prior knowledge about how pop music is composed. In particular, the bottom layers generate the melody, while the higher levels produce the drums and chords. We conduct several human studies that show strong preference of our generated music over that produced by the recent method by Google. We additionally show two applications of our framework: neural dancing and karaoke, as well as neural story singing.


Song From PI: A Musically Plausible Network for Pop Music Generation

Hang Chu, Raquel Urtasun, Sanja Fidler

[ICLR workshop]     [arXiv]     [Additional experiments]

Media Coverage

The Register The Guardian New York Post UofT News Nvidia News
Hacker News Metro News NPR News Berlin Radio The Culture Trip

Fan Performing:

Baca et al. (Vimeo)   Sea Level (Youtube)

Generated Songs:

Song 001: Song 002:
Song 003: Song 004:
Song 005: Song 006:
Song 007: Song 008:
Song 009: Song 010:

Neural Dancing and Karaoke:

Dancing-Karaoke 001:

Neural Dancing and Karaoke from Hang Chu on Vimeo.

Neural Story Singing

Story-Singing 001:

Neural Story Singing from Hang Chu on Vimeo.

Story-Singing 002: (Lyrics generation biased by a collection of 30 Christmas songs.)

Neural Story Singing Christmas from Hang Chu on Vimeo.

Vimeo Album of 12 Neural Story Singing Songs


Hang Chu
Raquel Urtasun
Sanja Fidler

University of Toronto