**For an up-to-date list see my Google Scholar page**

## Papers

- Distributed Second-order Optimization using Kronecker-Factored Approximations

Jimmy Ba, Roger Grosse,**James Martens**

In*International Conference on Learning Representations (ICLR)*, 2017

[PDF] - A Kronecker-factored approximate Fisher matrix for convolution layers

Roger Grosse,**James Martens**

In*Proceedings of the 33rd International Conference on Machine Learning (ICML)*, 2016

[arXiv] - Adding Gradient Noise Improves Learning for Very Deep Networks

Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach,**James Martens**

[arXiv] - Optimizing Neural Networks with Kronecker-factored Approximate Curvature

**James Martens**, Roger Grosse

In*Proceedings of the 32nd International Conference on Machine Learning (ICML)*, 2015

[Full version] [ICML version] [Appendices for ICML version] [Code] - New insights and perspectives on the natural gradient method

**James Martens**

[arXiv] - On the Expressive Efficiency of Sum Product Networks

**James Martens**, Venkatesh Medabalimi

[arXiv] - On the Representational Efficiency of Restricted Boltzmann Machines

**James Martens**, Arkadev Chattopadhyay, Toniann Pitassi, Richard Zemel

In*Proceedings of the 27th Annual Conference Neural Information Processing Systems (NIPS)*, 2013

[PDF] - On the importance of momentum and initialization in deep learning

Ilya Sutskever,**James Martens**, George Dahl, and Geoffery Hinton

In*Proceedings of the 30th International Conference on Machine Learning (ICML)*, 2013

[PDF] - Training Deep and Recurrent Neural Networks with Hessian-Free Optimization

**James Martens**, Ilya Sutskever

In*Neural Networks: Tricks of the Trade*, 2012

[PDF] [Publisher link (out of date version)] - Estimating the Hessian by Back-propagating Curvature

**James Martens**, Ilya Sutskever, and Kevin Swersky

In*Proceedings of the 29th International Conference on Machine Learning (ICML)*, 2012

[PDF] [Supplement/appendix] - Learning Recurrent Neural Networks with Hessian-Free Optimization

**James Martens**, Ilya Sutskever

In*Proceedings of the 28th International Conference on Machine Learning (ICML)*, 2011

[PDF] [Supplement] [Video] - Generating Text with Recurrent Neural Networks

Ilya Sutskever,**James Martens**, Geoffrey Hinton

In*Proceedings of the 28th International Conference on Machine Learning (ICML)*, 2011

[PDF] - Normalization for probabilistic inference with neurons

Chris Eliasmith,**James Martens**

In*Biological Cybernetics*, 2011

- Deep Learning via Hessian-free Optimization

**James Martens**

In*Proceedings of the 27th International Conference on Machine Learning (ICML)*, 2010

[PDF] [Slides] [Code] - Learning the Linear Dynamical System with ASOS

**James Martens**

In*Proceedings of the 27th International Conference on Machine Learning (ICML)*, 2010

[PDF] [Supplement and code] [Slides] - Parallelizable Sampling of Markov Random Fields

**James Martens**, Ilya Sutskever

In*Proceedings of Artificial Intelligence and Statistics (AISTATS)*, 2010

[PDF] - Novel Lead Configurations for Robust Bio-Impedance Acquisition

Joel Ironstone, Milan Graovac,**James Martens**, Martin Rozee, K.C. Smith

In*Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)*, 2007

## Theses

- Second-order Optimization for Neural Networks

**James Martens**

Ph.D. Thesis, Dept. of Computer Science, University of Toronto

[PDF] - A New Algorithm for Rapid Parameter Learning in Linear Dynamical Systems

**James Martens**

Master's Thesis, Dept. of Computer Science, University of Toronto

[PDF]

## Posters/Presentations

- A neurologically plausible implementation of statistical inference applied to random dot motion

**James Martens**, Chris Eliasmith

Submitted poster,*Computational Neuroscience*(CNS), 2007

- Challenges for biological bayes: Solving normalization

Chris Eliasmith,**James Martens**

Presentation,*COSYNE Workshop on Statistical Inference in the Brain*, 2007

- A biologically realistic model of statistical inference applied to random dot motion

**James Martens**, Chris Eliasmith

Submitted poster,*COSYNE*, 2007

## Patents

- Weighted gradient method and system for diagnosing disease.

Milan Graovac, James Martens, Zorin Pavlovic, Joel Ironstone

U.S. patent

## News :