Andriy Mnih

My email address can be easily derived from the URL for this page.

About me

I am a research scientist at DeepMind. Until February 2013, I was a postdoctoral researcher at Gatsby, working with Yee Whye Teh. Prior to that I was a PhD student in the Machine Learning Group at the University of Toronto, advised by Geoffrey Hinton.

Research interests

Latent variable models
Variational inference
Monte Carlo gradient estimation
Representation learning

Publications

Score Modeling for Simulation-based Inference

Workshop on Score-Based Methods, NeurIPS 2022 [arxiv]

Gaussian Dropout as an Information Bottleneck Layer
Mélanie Rey, Andriy Mnih
Bayesian Deep Learning Workshop, NeurIPS 2021 [paper]

Coupled Gradient Estimators for Discrete Latent Variables
Zhe Dong, Andriy Mnih, George Tucker
NeurIPS 2021 [arxiv]

Unbiased Gradient Estimation with Balanced Assignments for Mixtures of Experts
Wouter Kool, Chris J. Maddison, Andriy Mnih
I Can’t Believe It’s Not Better Workshop, NeurIPS 2021 [arxiv]

Generalized Doubly Reparameterized Gradient Estimators
Matthias Bauer, Andriy Mnih
ICML 2021 [arxiv]

The Lipschitz Constant of Self-Attention
Hyunjik Kim, George Papamakarios, Andriy Mnih
ICML 2021 [arxiv]

DisARM: An Antithetic Gradient Estimator for Binary Latent Variables
Zhe Dong, Andriy Mnih, George Tucker
NeurIPS 2020 [arxiv]

Monte Carlo Gradient Estimation in Machine Learning
Shakir Mohamed, Mihaela Rosca, Michael Figurnov, Andriy Mnih
Journal of Machine Learning Research 21 (132), 1-62 [JMLR]

Q-Learning in enormous action spaces via amortized approximate maximization
Tom Van de Wiele, David Warde-Farley, Andriy Mnih, Volodymyr Mnih
January 2020 [arxiv]

Sparse Orthogonal Variational Inference for Gaussian Processes
Jiaxin Shi, Michalis K. Titsias, Andriy Mnih
AISTATS 2020 [arxiv]

Measure-Valued Derivatives for Approximate Bayesian Inference
Mihaela Rosca, Michael Figurnov, Shakir Mohamed, Andriy Mnih
Bayesian Deep Learning Workshop, NeurIPS 2019 [pdf]

Attentive Neural Processes
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, Yee Whye Teh
ICLR 2019 [openreview] [arxiv]

Resampled Priors for Variational Autoencoders
Matthias Bauer, Andriy Mnih
AISTATS 2019 [arxiv]

Implicit Reparameterization Gradients
Michael Figurnov, Shakir Mohamed, Andriy Mnih
NeurIPS 2018 [NeurIPS] [arxiv]

Disentangling by Factorising
Hyunjik Kim, Andriy Mnih
ICML 2018 [arxiv] [pdf (workshop version)]

Continuous Relaxation Training of Discrete Latent Variable Image Models
Casper Kaae Sønderby, Ben Poole, Andriy Mnih
Bayesian Deep Learning Workshop, NIPS 2017 [pdf]

Variational Memory Addressing in Generative Models
Jörg Bornschein, Andriy Mnih, Daniel Zoran, Danilo J. Rezende
NIPS 2017 [arxiv]

REBAR : Low-variance, unbiased gradient estimates for discrete latent variable models
George Tucker, Andriy Mnih, Chris J. Maddison, Dieterich Lawson, Jascha Sohl-Dickstein
NIPS 2017 [arxiv]

Filtering Variational Objectives
Chris J. Maddison*, Dieterich Lawson*, George Tucker*, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, Yee Whye Teh
* denotes equal contribution
NIPS 2017 [arxiv]

The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
Chris J. Maddison, Andriy Mnih, Yee Whye Teh
ICLR 2017 [arxiv]

Variational inference for Monte Carlo objectives
Andriy Mnih and Danilo J. Rezende
ICML 2016 [arxiv] [slides] [poster] [bibtex]

MuProp: Unbiased Backpropagation for Stochastic Neural Networks
Shixiang Gu, Sergey Levine, Ilya Sutskever, Andriy Mnih
ICLR 2016 [arxiv]

Neural Variational Inference and Learning in Belief Networks
Andriy Mnih and Karol Gregor
ICML 2014 [pdf] [slides] [poster] [bibtex] [talk]

Deep AutoRegressive Networks
Karol Gregor, Ivo Danihelka, Andriy Mnih, Charles Blundell, Daan Wierstra
ICML 2014 [pdf] [bibtex]

Learning word embeddings efficiently with noise-contrastive estimation
Andriy Mnih and Koray Kavukcuoglu
NIPS 2013 [pdf] [poster] [bibtex]

Learning Label Trees for Probabilistic Modelling of Implicit Feedback
Andriy Mnih and Yee Whye Teh
NIPS 2012 [pdf] [poster] [bibtex]

A fast and simple algorithm for training neural probabilistic language models
Andriy Mnih and Yee Whye Teh
ICML 2012 [pdf] [slides] [poster] [bibtex] [5 min talk]

Taxonomy-Informed Latent Factor Models for Implicit Feedback
Andriy Mnih
JMLR W&CP Volume 18: Proceedings of KDD Cup 2011 [pdf] [slides] [bibtex]

Learning Distributed Representations for Statistical Language Modelling and Collaborative Filtering
Andriy Mnih
PhD Thesis, University of Toronto, 2009 [pdf] [bibtex]

Improving a Statistical Language Model Through Non-linear Prediction
Andriy Mnih, Zhang Yuecheng, and Geoffrey Hinton
Neurocomputing, 72:7-9, 2009 [bibtex]

A Scalable Hierarchical Distributed Language Model
Andriy Mnih and Geoffrey Hinton
NIPS 2008 [pdf] [bibtex]

Bayesian Probabilistic Matrix Factorization using Markov Chain Monte Carlo
Ruslan Salakhutdinov and Andriy Mnih
ICML 2008 [pdf] [bibtex]

Improving a Statistical Language Model by Modulating the Effects of Context Words
Zhang Yuecheng, Andriy Mnih, and Geoffrey Hinton
European Symposium on Artificial Neural Networks 2008 (ESANN 2008)

Probabilistic Matrix Factorization
Ruslan Salakhutdinov and Andriy Mnih
NIPS 2007 [pdf] [bibtex]

Three New Graphical Models for Statistical Language Modelling
Andriy Mnih and Geoffrey Hinton
ICML 2007 [pdf] [bibtex]

Restricted Boltzmann Machines for Collaborative Filtering
Ruslan Salakhutdinov, Andriy Mnih, and Geoffrey Hinton
ICML 2007 [pdf] [bibtex]

Visualizing Similarity Data with a Mixture of Maps
James Cook, Ilya Sutskever, Andriy Mnih, and Geoffrey Hinton
AISTATS 2007 [pdf] [bibtex]

Learning Nonlinear Constraints with Contrastive Backpropagation
Andriy Mnih and Geoffrey Hinton
International Joint Conference on Neural Networks 2005 (IJCNN 2005) [bibtex]

Wormholes Improve Contrastive Divergence
Geoffrey Hinton, Max Welling, and Andriy Mnih
NIPS 2003 [bibtex]