Multimodal Learning with Deep Boltzmann Machines
Nitish Srivastava, Ruslan Salakhutdinov
University of Toronto
Extended JMLR version [paper][bibtex]
Code for training deep models Deepnet
- Description of the features txt
- Preprocessed Data tar.gz [5.6 GB]
Reconstruction of multimodal queries
These were made by taking a multimodal query and reconstructing it after doing mean-field inference in the model.
Reconstructions/Retrieval from individual pathways
These reconstructions were made by going up and down the stack of RBMs used for pretraining the DBM.
- Replicated softmax model (Text model 1)
- 2 hidden layer DBN (Binary RBM on top of Text model 1) (Text model 2).
- Gaussian RBM (Image model 1).
- 2 hidden layer DBN (Binary RBM on top of Image model 1) (Image model 2).
Samples from conditional models
DeepNet model files and trainers
DBN Model files can be found here or
Model files for DBMs