Machine Learning Group
University of Toronto
Vector Institute
Canada

Email: gdzhang at cs dot toronto dot edu

Follow Me

"Many things are only trivial once you know them." - Herman Chernoff

About Me

I'm a PhD student in Machine Learning Group at University of Toronto, supervised by Roger Grosse.

Previously, I received my bachelor degree in Information Engineering from Zhejiang University in 2017. In 2016 summer, I was a visiting student at UCLA, where I worked with Song-Chun Zhu and Ying Nian Wu on energy-based generative models. After my visiting to UCLA, I spent half year interning at Microsoft Research Asia under the supervision of Dr. Jifeng Dai. In 2019, I worked as a student researcher at Google Brain Toronto (with Geoffrey Hinton and Lala Li). In early 2020, I was a visiting student at Institute of Advanced Study (IAS) for special-year program.

I'm currently working on Optimization and Generalization of Deep Learning and also Bayesian Deep Learning.

Email me if you have any questions about my paper, or if you're interested in collaborating with me.

Curriculum Vitae

My CV can be downloaded from this link: [CV].

Selected Publications (Google Scholar for Full List)

* below indicates equal contribution

Smooth Game Optimization

• Don’t Fix What ain’t Broke: Near-optimal Local Convergence of Alternating Gradient Descent-Ascent for Minimax Optimization
  Guodong Zhang, Yuanhao Wang, Laurent Lessard, Roger Grosse
  Arxiv, 2021.
  

• A Unified Analysis of First-Order Methods for Smooth Games via Integral Quadratic Constraints
  Guodong Zhang, Xuchan Bao, Laurent Lessard, Roger Grosse
  Journal of Machine Learning Research (JMLR), 2021.
  

• On the Suboptimality of Negative Momentum for Minimax Optimization
  Guodong Zhang, Yuanhao Wang
  International Conference on Artificial Intelligence and Statistics (AISTATS), 2021.
  

• On Solving Minimax Optimization Locally: A Follow-the-Ridge Approach
  Yuanhao Wang*, Guodong Zhang*, Jimmy Ba
  International Conference on Learning Representations (ICLR), 2020.
  

Neural Network Training Dynamics

• An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise
  Yeming Wen, Kevin Luk, Maxime Gazeau, Guodong Zhang, Harris Chan, Jimmy Ba
  International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
  

• Which Algorithmic Choices Matter at Which Batch Sizes?
  Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva,
  George E. Dahl, Christopher J. Shallue, Roger Grosse
  Neural Information Processing Systems (NeurIPS), 2019.
  

• Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks
  Guodong Zhang, James Martens, Roger Grosse
  Neural Information Processing Systems (NeurIPS), 2019.
  

• Three Mechanisms of Weight Decay Regularization
  Guodong Zhang, Chaoqi Wang, Bowen Xu, Roger Grosse
  International Conference on Learning Representations (ICLR), 2019.
  

Bayesian Deep Learning

• Functional Variational Bayesian Neural Networks
  Shengyang Sun*, Guodong Zhang*, Jiaxin Shi*, Roger Grosse
  International Conference on Learning Representations (ICLR), 2019.
  

• Differentiable Compositional Kernel Learning for Gaussian Processes
  Shengyang Sun, Guodong Zhang, Chaoqi Wang, Wenyuan Zeng, Jiaman Li, Roger Grosse
  International Conference on Machine Learning (ICML), 2018.
  

• Noisy Natural Gradient as Variational Inference
  Guodong Zhang*, Shengyang Sun*, David Duvenaud, Roger Grosse
  International Conference on Machine Learning (ICML), 2018.
  

Neural Network Pruning and Computer Vision

• Picking Winning Tickets Before Training by Preserving Gradient Flow
  Chaoqi Wang, Guodong Zhang, Roger Grosse
  International Conference on Learning Representations (ICLR), 2020.
  

• EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis
  Chaoqi Wang, Roger Grosse, Sanja Fidler, Guodong Zhang
  International Conference on Machine Learning (ICML), 2019.
  

• Deformable Convolutional Networks
  Jifeng Dai*, Haozhi Qi*, Yuwen Xiong*, Yi Li*, Guodong Zhang*, Han Hu, Yichen Wei
  IEEE International Conference on Computer Vision (ICCV), 2017.
  

Teaching

• CSC 2541: Neural Network Training Dynamics (2021 Winter)
• CSC 311: Introduction to Machine Learning (2020 Fall)
• CSC 2515: Machine Learning (2019 Fall)
• CSC 411: Machine Learning and Data Mining (2018 Fall)
• CSC 321: Introduction to Neural Networks and Machine Learning (2018 Winter)
• CSC 384: Introduction to Artificial Intelligence (2017 Fall, 2018 Summer, 2019 Winter)

Services

• Conference Reviewer: UAI, AISTATS, NeurIPS, ICLR, ICML
• Journal Reviewer: JMLR

Selected Honors & Awards