George E. Dahl

email: Can be easily derived from the URL for this page.

About me

I am a PhD Student in the Machine Learning Group, supervised by Geoffrey Hinton.

Context-Dependent Pre-trained Deep Neural Networks for Large Vocabulary Speech Recognition

IEEE Transactions on Audio, Speech, and Language Processing

Large Vocabulary Continuous Speech Recognition with Context-Dependent DBN-HMMs

ICASSP 2011

Deep Belief Networks Using Discriminative Features for Phone Recognition

ICASSP 2011

Acoustic Modeling using Deep Belief Networks

IEEE Trans. on Audio, Speech and Language Processing.

Deep Belief Networks for Phone Recognition

NIPS Workshop on Deep Learning for Speech Recognition and Related Applications

Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine

Advances in Neural Information Processing Systems 23

Incorporating Side Information into Probabilistic Matrix Factorization Using Gaussian Processes

Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence

I have implemented a version of the Hessian Free (truncated Newton) optimization approach that is based on James Martens' exposition of it in his paper that explored using HF for deep learning (please see James Martens' research page). My particular implementation was made possible with Ilya Sutskever's guidance and some of the implementation choices have been made to make it easier to compare my code to various optimizers he has written. Despite Ilya's generous assistance, any bugs or defects that might exist in the code I post here are my own. Please see Ilya's publication page for code he has released for HF and recurrent neural nets. It isn't too difficult to wrap his recurrent neural net model code in a way that let's my optimizer code optimize it. Without further ado, here is the code. The file is large because it also contains a copy of the curves dataset. The code requires gnumpy to run and I recommend using cudamat , written by Volodymyr Mnih , and running the code on a GPU and not in the slower simulation mode of gnumpy.