Generalizing from MIDI to Real-World Audio

To further explore the generalization capability of TimbreTron, we also tried one domain adaptation experiment where we took a CycleGAN trained on MIDI data, tested it on the real world test dataset, and synthesized audio with Wavenet trained on training real world data. To test generalization ability of TimbreTron, we conducted experiments where SpecGAN is trained on unpaired MIDI data, and then evaluated on real world data test set. The WaveNet syntheiszer here is trained on real world data.

Examples of Piano Samples from MIDI training Dataset

Examples of Harpsichord Samples from MIDI training Dataset

Samples Generated by TimbreTron trained on MIDI but tested on Real World test Dataset(Piano pieces played by Sageev)

1.Source Piano

2. Source Piano

3. Source Piano

1.Generated Harpsichord

2. Generated Harpsichord

3. Generated Harpsichord

4. Source Piano

5. Source Piano

4. Generated Harpsichord

5. Generated Harpsichord

As is shown from the corresponding audio examples in this section, the quality of generated audio is very good, with pitch preserved and timbre transferred. The ability to generalize from MIDI to real-world is interesting, in that it opens up the possibility of training on paired examples.