Renjie Liao  

I'm a PhD candidate in Department of Computer Science, University of Toronto.
I also work at Uber Advanced Technology Group and Vector Institute.

My research interests are machine learning, computer vision, and self-driving.
I recently focused on deep (probabilistic) models of (graph) structured data and their applications. Highlights include:


My advisors are Richard Zemel and Raquel Urtasun. My M.Phil. advisor was Jiaya Jia at the Chinese University of Hong Kong, where I worked mostly on computer vision. I got B.Eng. degree from Beihang University. I did internships at Microsoft Research Cambridge and Asia. I had the good fortune of working with some smart "neural networks".

Curriculum Vitae
Email: rjliao at cs dot toronto dot edu



"There is nothing as practical as a good theory." - Kurt Lewin
"The test of all knowledge is experiment." - Richard Feynman

Selected Publications

* below indicates equal contribution
Link
Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
Sergio Casas, Cole Gulino, Renjie Liao, Raquel Urtasun
International Conference on Robotics and Automation (ICRA), 2020
Link
Efficient Graph Generation with Graph Recurrent Attention Networks
Renjie Liao, Yujia Li, Yang Song, Shenlong Wang, Charlie Nash, William L. Hamilton, David Duvenaud, Raquel Urtasun, Richard S. Zemel
Neural Information Processing Systems (NeurIPS), 2019

[Code]

Link
Incremental Few-Shot Learning with Attention Attractor Networks
Mengye Ren, Renjie Liao, Ethan Fetaya, Richard S. Zemel
Neural Information Processing Systems (NeurIPS), 2019

[Code]

Link
DMM-Net: Differentiable Mask-Matching Network for Video Object Segmentation
Xiaohui Zeng*, Renjie Liao*, Li Gu, Yuwen Xiong, Sanja Fidler, Raquel Urtasun
IEEE International Conference on Computer Vision (ICCV), 2019

[Code] [Video]

Link
Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction
Ajay Jain*, Sergio Casas Romero*, Renjie Liao*, Yuwen Xiong*, Song Feng, Sean Segal, Raquel Urtasun
Conference on Robot Learning (CoRL), 2019
Link
Lorentzian Distance Learning for Hyperbolic Representations
Marc T. Law, Renjie Liao, Jake Snell, Richard S. Zemel
International Conference on Machine Learning (ICML), 2019

[Code]

Link
Inference in Probabilistic Graphical Models by Graph Neural Networks
KiJung Yoon, Renjie Liao, Yuwen Xiong, Lisa Zhang, Ethan Fetaya, Raquel Urtasun, Richard S. Zemel, Xaq Pitkow
ICML Workshop on Tractable Probabilistic Modeling, 2019

[ICLR 2018 Workshop][Code Coming Soon!] [Best Paper Award]

Link
UPSNet: A Unified Panoptic Segmentation Network
Yuwen Xiong*, Renjie Liao*, Hengshuang Zhao*, Rui Hu, Min Bai, Ersin Yumer, Raquel Urtasun
International Conference on Computer Vision and Pattern Recognition (CVPR), 2019

[Code] [Oral Presentation, 288/5160 (5.6%)]

Link
DARNet: Deep Active Ray Network for Building Segmentation
Dominic Cheng, Renjie Liao, Sanja Fidler, Raquel Urtasun
International Conference on Computer Vision and Pattern Recognition (CVPR), 2019

[Code]

Link
LanczosNet: Multi-Scale Deep Graph Convolutional Networks
Renjie Liao, Zhizhen Zhao, Raquel Urtasun, Richard S. Zemel
International Conference on Learning Representations (ICLR), 2019

[Code] [NeurIPS 2018 R2L Workshop] [Score Rank: 69/1591 (4.4%)]

Link
Neural Guided Constraint Logic Programming for Program Synthesis
Lisa Zhang, Gregory Rosenblatt, Ethan Fetaya, Renjie Liao, William E. Byrd, Matthew Might, Raquel Urtasun, Richard S. Zemel
Neural Information Processing Systems (NeurIPS), 2018

[Code] [ICLR 2018 Workshop]

Link
Reviving and Improving Recurrent Back-Propagation
Renjie Liao*, Yuwen Xiong*, Ethan Fetaya, Lisa Zhang, KiJung Yoon, Xaq Pitkow, Raquel Urtasun, Richard S. Zemel
International Conference on Machine Learning (ICML), 2018

[Code] [Video] [Full Oral Presentation, 212/2473 (8.6%)]

Link
Graph Partition Neural Networks for Semi-Supervised Classification
Renjie Liao, Marc Brockschmidt, Daniel Tarlow, Alexander Gaunt, Raquel Urtasun, Richard S. Zemel
International Conference on Learning Representations Workshop (ICLR), 2018

[Code]

Link
NerveNet: Learning Structured Policy with Graph Neural Networks
Tingwu Wang*, Renjie Liao*, Jimmy Ba, Sanja Fidler
International Conference on Learning Representations (ICLR), 2018

[Project] [Code] [Video]

Link
Understanding Short-Horizon Bias in Stochastic Meta-Optimization
Yuhuai Wu, Mengye Ren, Renjie Liao, Roger Grosse
International Conference on Learning Representations (ICLR), 2018

[Code]

Link
GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation
Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, Jiaya Jia
International Conference on Computer Vision and Pattern Recognition (CVPR), 2018

[Code]

Link
Learning Deep Structured Active Contours End-to-End
Diego Marcos, Devis Tuia, Benjamin Kellenberger, Lisa Zhang, Min Bai, Renjie Liao, Raquel Urtasun
International Conference on Computer Vision and Pattern Recognition (CVPR), 2018

[Code] [Spotlight Presentation, 224/3303 (6.8%)]

Link
3D Graph Neural Networks for RGBD Semantic Segmentation
Xiaojuan Qi, Renjie Liao, Jiaya Jia, Sanja Fidler, Raquel Urtasun
IEEE International Conference on Computer Vision (ICCV), 2017

[Code] [ PyTorch Implementation ] [Oral Presentation, 45/2143 (2.1%)]

Link
Situation Recognition with Graph Neural Networks
Ruiyu Li, Makarand Tapaswi, Renjie Liao, Jiaya Jia, Raquel Urtasun, Sanja Fidler
IEEE International Conference on Computer Vision (ICCV), 2017

[Code]

Link
Detail-revealing Deep Video Super-Resolution
Xin Tao, Hongyun Gao, Renjie Liao, Jue Wang, Jiaya Jia
IEEE International Conference on Computer Vision (ICCV), 2017

[Code] [Oral Presentation, 45/2143 (2.1%)]

Link
Normalizing the Normalizers: Comparing and Extending Network Normalization Schemes
Mengye Ren*, Renjie Liao*, Raquel Urtasun, Fabian H. Sinz, Richard S. Zemel
International Conference on Learning Representations (ICLR), 2017

[Code]

Link
Learning to Generate Images with Perceptual Similarity Metrics
Jake Snell, Karl Ridgeway, Renjie Liao, Brett D. Roads, Michael C. Mozer, Richard S. Zemel
International Conference on Image Processing (ICIP), 2017

Link
Learning Deep Parsimonious Representation
Renjie Liao, Alexander Schwing, Richard S. Zemel, Raquel Urtasun
Neural Information Processing Systems (NIPS), 2016

[Code]

Link
Video Super-Resolution via Deep Draft-Ensemble Learning
Renjie Liao, Xin Tao, Ruiyu Li, Ziyang Ma, Jiaya Jia
IEEE International Conference on Computer Vision (ICCV), 2015

[Project & Code]

Link
Semantic Segmentation With Object Clique Potential
Xiaojuan Qi, Jianping Shi, Shu Liu, Renjie Liao, Jiaya Jia
IEEE International Conference on Computer Vision (ICCV), 2015

Link
Handling Motion Blur in Multi-Frame Super-Resolution
Ziyang Ma, Renjie Liao, Xin Tao, Li Xu, Jiaya Jia, Enhua Wu
International Conference on Computer Vision and Pattern Recognition (CVPR), 2015

[Project & Code]

Link
Deep Edge-Aware Filters
Li Xu, Jimmy Ren, Qiong Yan, Renjie Liao, Jiaya Jia
International Conference on Machine Learning (ICML), 2015

[Code]

Link
Nonparametric Bayesian Upstream Supervised Multi-Modal Topic Models
Renjie Liao, Jun Zhu, Zengchang Qin
ACM International Conference on Web Search and Data Mining (WSDM), 2014

Link
Learning Important Spatial Pooling Regions for Scene Classification
Di Lin, Cewu Lu, Renjie Liao, Jiaya Jia
International Conference on Computer Vision and Pattern Recognition (CVPR), 2014

Link
CoDeL: An Efficient Human Co-detection and Labeling Framework
Jianping Shi*, Renjie Liao*, Jiaya Jia
IEEE International Conference on Computer Vision (ICCV), 2013

[Project]

Link
Image Super-Resolution Using Local Learnable Kernel Regression
Renjie Liao, Zengchang Qin
Asian Conference on Computer Vision (ACCV), 2012

[Code]

Preprints

Link
Latent Variable Modelling with Hyperbolic Normalizing Flows
Avishek Joey Bose, Ariella Smofsky, Renjie Liao, Prakash Panangaden, William L. Hamilton
arXiv preprint arXiv:2002.06336 (2020)

Link
Nonlinear Equation Solving: A Faster Alternative to Feedforward Computation
Yang Song, Chenlin Meng, Renjie Liao, Stefano Ermon
arXiv preprint arXiv:2002.03629 (2020)

Link
Deformable Filter Convolution for Point Cloud Reasoning
Yuwen Xiong, Mengye Ren, Renjie Liao, Kelvin Wong, Raquel Urtasun
arXiv preprint arXiv:1907.13079 (2019)

Link
Alchemy: A Quantum Chemistry Dataset for Benchmarking AI Models
Guangyong Chen, Pengfei Chen, Chang-Yu Hsieh, Chee-Kong Lee, Benben Liao, Renjie Liao, Weiwen Liu, Jiezhong Qiu, Qiming Sun, Jie Tang, Richard S. Zemel, Shengyu Zhang
arXiv preprint arXiv:1906.09427 (2019)

[Dataset + Competition]

Link
Bounded-Distortion Metric Learning
Renjie Liao, Jianping Shi, Ziyang Ma, Jun Zhu, Jiaya Jia.
arXiv preprint arXiv:1505.02377 (2015)

[Code]

Talks

Teaching

Services

Honors & Awards

Best Reviewer, NeurIPS 2019
RBC Graduate Fellowship, 2019
Connaught International Scholarship for Doctoral Students (University-wide 20), University of Toronto, 2015.
Departmental Entrance Scholarship (Top %2 applicant), Department of Computer Science, University of Toronto, 2015.
3rd prize (1%), China Finals in Software Design Competition of Imagine Cup, Microsoft, 2010.
1st prize (10 out of candidates of all majors), The 20th Feng Ru Cup, Beihang University, 2010.
Travel Award: ICLR 2019, ICML 2018, ICLR 2017, ACCV 2012, MLSS 2011.

Miscellany

Hobbies: Fingerstyle Guitar, Basketball, Soccer, PC Games.

My Erdös number is 4 (F. Sinz = 3, G. Rätsch = 2, A. Jagota = 1, P. Erdös = 0).

Books I have read/am reading/will read.

Last Updated by Renjie: Apr. 17, 2020