George E. Dahl
PhD Student
Machine Learning Group
Department of Computer Science
University of Toronto
Ontario, Canada
email: Can be easily derived from the URL for this page.
About me
I am a PhD Student in the Machine Learning Group, supervised by Geoffrey Hinton.
Research interests
- deep learning architectures
- speech recognition and language processing
- undirected graphical models
- most of statistical machine learning
Selected Publications
Context-Dependent Pre-trained Deep Neural Networks for Large Vocabulary Speech Recognition
George E. Dahl, Dong Yu, Li Deng, and Alex Acero
Accepted for publication in IEEE Transactions on Audio, Speech, and Language Processing
[pdf draft]
[bibtex coming soon]
This paper is a draft, check back later for the final version.
Large Vocabulary Continuous Speech Recognition with Context-Dependent DBN-HMMs
George E. Dahl, Dong Yu, Li Deng, and Alex Acero
Accepted for publication in ICASSP 2011
[pdf]
[bibtex]
This paper is a conference-length version of the journal paper listed immediately above.
Deep Belief Networks Using Discriminative Features for Phone Recognition
Abdel-rahman Mohamed, Tara N. Sainath, George E. Dahl, Bhuvana Ramabhadran, Geoffrey E. Hinton, and Michael A. Picheny
Accepted for publication in ICASSP 2011
[pdf]
[bibtex]
Acoustic Modeling using Deep Belief Networks
Abdel-rahman Mohamed, George E. Dahl, and Geoffrey E. Hinton
Accepted for publication in IEEE Trans. on Audio, Speech and Language Processing.
[pdf]
[bibtex coming soon]
Deep Belief Networks for Phone Recognition
Abdel-rahman Mohamed, George E. Dahl, Geoffrey E. Hinton
In NIPS Workshop on Deep Learning for Speech Recognition and Related Applications, 2009.
[pdf]
[bibtex]
The journal version of this work (listed immediately above) should be viewed as the definitive version.
Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine
George E. Dahl, Marc'Aurelio Ranzato, Abdel-rahman Mohamed, and Geoffrey E. Hinton
In Advances in Neural Information Processing Systems
23, 2010.
[pdf]
[bibtex]
Incorporating Side Information into Probabilistic Matrix Factorization Using Gaussian Processes
Ryan Prescott Adams, George E. Dahl, and Iain Murray
In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence, 2010.
[pdf]
[bibtex]
[code]
Code
I have implemented a version of the Hessian Free
(truncated Newton) optimization approach that is based on James
Martens' exposition of it in his paper that explored using HF for deep
learning (please see James Martens'
research page). My
particular implementation was made possible with Ilya
Sutskever's guidance and some of the implementation choices have
been made to make it easier to compare my code to various optimizers
he has written. Despite Ilya's generous assistance, any bugs or
defects that might exist in the code I post here are my own. Please
see Ilya's publication
page for code he has released for HF and recurrent neural nets. It
isn't too difficult to wrap his recurrent neural net model code in a
way that let's my optimizer code optimize it. Without further
ado, here is the code. The file is large
because it also contains a copy of the curves dataset. The code
requires gnumpy
to run and I recommend
using cudamat, written
by Volodymyr Mnih, and
running the code on a GPU and not in the slower simulation mode of
gnumpy.