Publications


Some of the papers are released with code, which comes with a license.

All papers that use GPUs are implemented with gnumpy, which is built on of cudamat


Conference and Journal papers

• Training Recurrent Neural Networks with Hessian-Free Optimization,
  James Martens and Ilya Sutskever, ICML 2011.
  [pdf, supplement, video, code]
  


• Generating Text with Recurrent Neural Networks,
  Ilya Sutskever, James Martens, and Geoffrey Hinton, ICML 2011.
  [pdf, recorded talk, code]
  


• Parallelizable Sampling for MRFs,
  James Martens and Ilya Sutskever, AISTATS 2010.
  [pdf]
  


• On the convergence properties of Contrastive Divergence,
  Ilya Sutskever and Tijmen Tieleman, AISTATS 2010.
  [pdf]
  


• Modelling Relational Data using Bayesian Clustered Tensor Factorization,
  Ilya Sutskever, Ruslan Salakhutdinov, and Josh Tenenbaum, NIPS*22, 2009.
  [pdf]
  


• A simpler unified analysis of Budget Perceptrons,
  Ilya Sutskever, ICML 2009.
  [pdf, recorded talk]
  


• Using matrices to model symbolic relationship,
  Ilya Sutskever and Geoffrey Hinton, NIPS*21, 2008.
  [pdf, code, matlab code (by Laurens van der Maaten)]
  


• The Recurrent Temporal Restricted Boltzmann Machine,
  Ilya Sutskever, Geoffrey Hinton and Graham Taylor, NIPS*21, 2008.
  [pdf, code, videos]
  


• Temporal Kernel Recurrent Neural Networks,
  Ilya Sutskever and Geoffrey Hinton, Neural Networks, Vol. 23, Issue 2, March 2010, Pages 239-243.
  [online text, code; but note that the idea was invented much earlier, 1, 2]


• Mimicking Go Experts with Convolutional Neural Networks,
  Ilya Sutskever and Vinod Nair, ICANN 2008.
  [pdf]
  


• Deep Narrow Sigmoid Belief Networks are Universal Approximators,
  Ilya Sutskever and Geoffrey Hinton, Neural Computation, November 2008, Vol. 20, No. 11: 2629-2636.
  [pdf]
  


• Visualizing Similarity Data with a Mixture of Maps,
  James Cook, Ilya Sutskever, Andriy Mnih, and Geoffrey Hinton, AISTATS 2007.
  [pdf]
  


• Learning Multilevel Distributed Representations for High-Dimensional Sequences,
  Ilya Sutskever and Geoffrey Hinton, AISTATS 2007.
  [pdf, videos, code]
  

Theses

• Nonlinear Multilayered Sequence Models,
  Ilya Sutskever, Master's Thesis, 2007.
  [link]

Cool stuff

• Trans-lingual representation of text documents,
  John Platt and Ilya Sutskever,
  Patent #20100324883
  

• The vectorizer
  Automatically transforms easy-to-read mathematical expressions into vectorized matlab code, 2003. Coded in Haskell.
  [link]

home