Multimodal Learning with Deep Boltzmann Machines

Nitish Srivastava, Ruslan Salakhutdinov
University of Toronto

[paper] [supplementary material] [poster] [video] [Presentation]
Extended JMLR version [paper][bibtex]

Code

Code for training deep models Deepnet

Data

Results

Reconstruction of multimodal queries

These were made by taking a multimodal query and reconstructing it after doing mean-field inference in the model.

Reconstructions/Retrieval from individual pathways

These reconstructions were made by going up and down the stack of RBMs used for pretraining the DBM.
Text
Image

Samples from conditional models

DeepNet model files and trainers

DBN Model files can be found here or on GitHub
Model files for DBMs