In the following, we first list some papers published since 2008, to reflect the new research activities since the last deep learning workshop held at NIPS, Dec 2007, and then list some earlier papers as well.
Papers published since 2008
- Analysis and understanding of deep belief networks [Le Roux & Bengio, 2008,Salakhutdinov & Murray, 2008]
- Biologically-inspired models [Karklin & Lewicki, 2008]
- Natural language processing using multi-task deep neural networks [Collobert & Weston, 2008]
- Deep neural networks with semi-supervised embedding [Weston et al., 2008]
- Deep neural networks by transfer learning from side features [Ahmed et al., 2008,Yu et al., 2009]
- Document and image retrieval using semi-supervised deep models [Ranzato & Szummer, 2008,Torralba et al., 2008]
- Learning robust features with denoising autoencoders [Vincent et al., 2008]
- Learning Gaussian process covariance by deep belief networks [Salakhutdinov & Hinton, 2008]
- Sparse feature learning with deep belief networks [Ranzato et al., 2008,Lee et al., 2008]
- Hierarchical distributed language model [Mnih & Hinton, 2009].
Papers published before 2008
- A paper introducing deep belief networks as generative models [Hinton et al., 2006]
- A book chapter about the philosophy behind models with deep architectures, motivating them in the context of Artificial Intelligence [Bengio & LeCun, 2007]
- A novel way of using greedy layer-wise learning for Convolutional Networks [Ranzato et al., 2007]
- General study of the framework of initializing a deep feed-forward neural network using a greedy layer-wise procedure [Hinton, 2006,Bengio et al., 2007,Bengio, 2009]
- An application of greedy layer-wise learning to dimensionality reduction and information retrieval [Hinton & Salakhutdinov, 2006,Salakhutdinov & Hinton, 2007]
- An evaluation of deep networks on numerous datasets related to vision: [Larochelle et al., 2007]
- Generalizing Restricted Boltzmann Machines to exponential family distributions. [Welling et al., 2004]
Bibliography
- Ahmed et al., 2008
-
Ahmed, A., Yu, K., Xu, W., Gong, Y., & Xing, E. P. (2008).
Training hierarchical feed-forward visual recognition models using transfer learning from pseudo tasks.
European Conference on Computer Vision. pdf - Bengio, 2009
-
Bengio, Y. (2009).
Learning deep architectures for AI.
To appear in Foundations and Trends in Machine Learning. pdf - Bengio et al., 2007
-
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007).
Greedy layer-wise training of deep networks.
Neural Information Processing Systems. pdf - Bengio & LeCun, 2007
-
Bengio, Y., & LeCun, Y. (2007).
Scaling learning algorithms towards AI.
Large Scale Kernel Machines. pdf - Collobert & Weston, 2008
-
Collobert, R., & Weston, J. (2008).
A unified architecture for natural language processing: Deep neural networks with multitask learning.
International Conference on Machine Learning. pdf - Hinton, 2006
-
Hinton, G. (2006).
To recognize shapes, first learn to generate images.
Technical Report. pdf - Hinton et al., 2006
-
Hinton, G., Osindero, S., & Teh., Y.-W. (2006).
A fast learning algorithm for deep belief nets.
Neural Computation, 18, 1527-1554. pdf - Hinton & Salakhutdinov, 2006
-
Hinton, G. E., & Salakhutdinov, R. R. (2006).
Reducing the dimensionality of data with neural networks.
Science, 313, 504 - 507. pdf - Karklin & Lewicki, 2008
-
Karklin, Y., & Lewicki, M. (2008).
Emergence of complex cell properties by learning to generalize in natural scenes.
Nature. link - Larochelle et al., 2007
-
Larochelle, H., Erhan, D., Courville, A., Bergstra, J., & Bengio, Y. (2007).
An empirical evaluation of deep architectures on problems with many factors of variation.
ICML'07. pdf - Le Roux & Bengio, 2008
-
Le Roux, N., & Bengio, Y. (2008).
Representational power of restricted Boltzmann machines and deep belief networks.
Neural Computation. pdf - Lee et al., 2008
-
Lee, H., Ekanadham, C., & Ng, A. Y. (2008).
Sparse deep belief network model for visual area V2.
Neural Information Processing Systems. pdf - Lee et al., 1998
-
Lee, T., Mumford, D., Romero, R., & Lamme, V. (1998).
The role of the primary visual cortex in higher level vision.
Vision research, 38, 2429-2454. pdf - Mnih & Hinton, 2009
-
Mnih, A., & Hinton, G. (2009).
A scalable hierarchical distributed language model.
Neural Information Processing Systems. pdf - Ranzato et al., 2008
-
Ranzato, M., Boureau, Y.-L., & LeCun, Y. (2008).
Sparse feature learning for deep belief networks.
Neural Information Processing Systems. pdf - Ranzato et al., 2007
-
Ranzato, M., Huang, F.-J., Boureau, Y.-L., & LeCun, Y. (2007).
Unsupervised learning of invariant feature hierarchies with applications to object recognition.
IEEE Conference on Computer Vision and Pattern Recognition. pdf - Ranzato & Szummer, 2008
-
Ranzato, M., & Szummer, M. (2008).
Semi-supervised learning of compact document representations with deep networks.
International Conferenece on Machine Learning. pdf - Salakhutdinov & Hinton, 2007
-
Salakhutdinov, R., & Hinton, G. (2007).
Semantic hashing.
SIGIR workshop on Information Retrieval and applications of Graphical Models. pdf - Salakhutdinov & Hinton, 2008
-
Salakhutdinov, R., & Hinton, G. (2008).
Using deep belief nets to learn covariance kernels for Gaussian Processes.
Neural Information Processing Systems. pdf - Salakhutdinov & Murray, 2008
-
Salakhutdinov, R., & Murray, I. (2008).
On the quantitative analysis of deep belief networks.
International Conference on Machine Learning. pdf - Torralba et al., 2008
-
Torralba, A., Fergus, R., & Weiss, Y. (2008).
Small codes and large image databases for recognition.
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pdf - Vincent et al., 2008
-
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008).
Extracting and composing robust features with denoising autoencoders.
International Conference on Machine Learning. pdf - Welling et al., 2004
-
Welling, M., Rosen-Zvi, M., & Hinton, G. E. (2004).
Exponential family harmoniums with an application to information retrieval.
NIPS. pdf - Weston et al., 2008
-
Weston, J., Ratle, F., & Collobert, R. (2008).
Deep learning via semi-supervised embedding.
International Conference on Machine Learning. pdf - Yu et al., 2009
-
Yu, K., Xu, W., & Gong, Y. (2009).
Deep learning with kernel regularization for visual recognition.
Neural Information Processing Systems. pdf