Jingkang Wang

Jingkang Wang    

PhD student
Machine Learning Group
Computer Science
University of Toronto

Email:   wangjk (at) cs (dot) toronto (dot) edu

LinkedIn  /  Twitter  /  GitHub  /  Google Scholar  /  CV

Bio

I am a fifth-year Ph.D. student in Machine Learning Group, CS Department, University of Toronto. My advisor is Prof. Raquel Urtasun. I am also part of Vector Institute and Waabi. From 2019-2021, I was also a research scientist at Uber-ATG Toronto, working with Prof. Raquel Urtasun. I received my B.S. degree from Shanghai Jiao Tong University working with Prof. Cewu Lu and Prof. Gongshen Liu. Prior to UofT, I spent a wonderful year working with Prof. Bo Li at UIUC as a research intern.

I am interested in 3D vision and robotics, in particular developing simulation systems to capture the long-tail scenarios that rarely happen in the real world for safe self-driving vehicles. The goal is to build realistic, robust and scalable simulation systems for robots. Then we leverage them for comprehensive testing or robustness enhancement of full autonomy stack for safe deployment in real world.

News

  • NEW 2023/08: LightSim is accepted at NeurIPS 2023. Congrats to Ava and Gary! Checkout our our latest neural lighting simulation system for self-driving.
  • NEW 2023/08: Adv3D is accepted at CoRL 2023. Congrats to Jay!
  • NEW 2023/07: Two papers are accepted at ICCV 2023: neural scene rasterization, lidar sim domain gap.
  • NEW 2023/06: Waabi research hub online, check websites for UniSim, UltraLiDAR, NeuSim and CADSim!
  • NEW 2023/03: UniSim and UltraLiDAR are accepted at CVPR 2023. Check our latest works in closed-loop sensor simulation (LiDAR + Camera with full controllability) and LiDAR generation!
  • 2023/01: NeuSim is accepted at ICRA 2023.
  • 2022/10: CADSim is accepted at CoRL 2022, excited to meet in New Zealand!
  • 2021/11: 2021 Baidu Fellowship Finalist.
  • 2021/10: One paper is accepted at CoRL 2021.
  • 2021/09: Two papers are accepted at NeurIPS 2021: min-max attack, peer loss in RL/IL.
  • 2021/07: One paper is accepted at ICCV 2021.
  • 2021/06: Give a talk (Safety-critical scenario generation for full autonomy testing) in CVPR21 tutorial.
  • 2021/03: Joined Waabi to work on sensor simulation for safe self driving.
  • 2021/03: AdvSim is accepted at CVPR 2021. Check out our framework for mixed-reality LiDAR simulation and safety-critical scenario generation with full autonomy involved.

    • Publications   (show selected)

    LightSim: Neural Lighting Simulation for Urban Scenes
    Ava Pun*, Gary Sun*, Jingkang Wang*, Yun Chen, Ze Yang, Sivabalan Manivasagam, Wei-Chiu Ma, Raquel Urtasun
    Advances in Neural Information Processing Systems (NeurIPS), 2023
    [paper][video][website]

    Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation
    Jay Sarva, Jingkang Wang, James Tu, Yuwen Xiong, Sivabalan Manivasagam, Raquel Urtasun
    Conference on Robot Learning (CoRL), 2023
    [paper][video][website]

    Towards Zero Domain Gap: A Comprehensive Study of Realistic LiDAR Simulation
    for Autonomy Testing
    Sivabalan Manivasagam*, Ioan Andrei Bârsan*, Jingkang Wang, Ze Yang, Raquel Urtasun
    International Conference on Computer Vision (ICCV), 2023
    [paper][video][website]

    Real-Time Neural Scene Rasterization for Large Scenes
    Jeffrey Liu, Yun Chen*, Ze Yang*, Jingkang Wang, Sivabalan Manivasagam, Raquel Urtasun
    International Conference on Computer Vision (ICCV), 2023
    [paper][video][website]

    UniSim: A Neural Closed-Loop Sensor Simulator
    Ze Yang*, Yun Chen*, Jingkang Wang*, Sivabalan Manivasagam*, Wei-Chiu Ma,
    Anqi Joyce Yang, Raquel Urtasun
    Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)
    [paper] [video] [poster] [4K demo] [website]

    UltraLiDAR: Learning Compact Representations for LiDAR Completion and Generation
    Yuwen Xiong, Wei-Chiu Ma, Jingkang Wang, Raquel Urtasun
    Conference on Computer Vision and Pattern Recognition (CVPR), 2023
    [paper] [video] [poster] [website]

    NeuSim: Reconstructing Objects in-the-wild for Realistic Sensor Simulation
    Ze Yang, Sivabalan Manivasagam, Yun Chen, Jingkang Wang, Rui Hu, Raquel Urtasun
    IEEE International Conference on Robotics and Automation (ICRA), 2023
    [paper] [video] [website]

    CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Simulation
    Jingkang Wang, Sivabalan Manivasagam, Yun Chen, Ze Yang, Ioan Andrei Bârsan, Anqi Joyce Yang,
    Wei-Chiu Ma, Raquel Urtasun
    Conference on Robot Learning (CoRL), 2022
    [paper] [video] [website]

    AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
    Jingkang Wang, Ava Pun, James Tu, Sivabalan Manivasagam, Abbas Sadat, Sergio Casas,
    Mengye Ren, Raquel Urtasun
    Conference on Computer Vision and Pattern Recognition (CVPR), 2021
    [paper] [poster] [video] [website]

    Adversarial Attack Generation Empowered by Min-Max Optimization
    Jingkang Wang*, Tianyun Zhang*, Sijia Liu, Pin-Yu Chen, Jiacen Xu, Makan Fardad, Bo Li
    Advances in Neural Information Processing Systems (NeurIPS), 2021
    [paper] [code] [poster] [website]

    Policy Learning Using Weak Supervision
    Jingkang Wang*, Hongyi Guo*, Zhaowei Zhu*, Yang Liu
    Advances in Neural Information Processing Systems (NeurIPS), 2021
    [paper] [code] [poster] [website]

    Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes
    Sean Segal*, Nishanth Kumar*, Sergio Casas, Wenyuan Zeng, Mengye Ren, Jingkang Wang, Raquel Urtasun
    Conference on Robot Learning (CoRL), 2021
    [paper]

    Adversarial Attacks on Multi-Agent Communication
    James Tu*, Tsunhsuan Wang*, Jingkang Wang, Sivabalan Manivasagam, Mengye Ren, Raquel Urtasun
    International Conference on Computer Vision (ICCV), 2021
    [paper] [video]

    Learning to Communicate and Correct Pose Errors
    Nicholas Vadivelu, Mengye Ren, James Tu, Jingkang Wang, Raquel Urtasun
    Conference on Robot Learning (CoRL), 2020
    [paper] [video]

    Cost-Efficient Online Hyperparameter Optimization
    Jingkang Wang*, Mengye Ren*, Ilija Bogunovic, Yuwen Xiong, Raquel Urtasun
    International Conference on Machine Learning (ICML), RealML Workshop, 2020
    arXiv preprint arXiv:2101.06590
    [paper] [video]

    Zero-Shot Compositional Policy Learning via Language Grounding
    Tianshi Cao*, Jingkang Wang*, Annie Zhang, Sivabalan Manivasagam
    International Conference on Learning Representations (ICLR), BeTR-RL Workshop, 2020
    [paper] [code] [media]

    Reinforcement Learning with Perturbed Rewards
    Jingkang Wang, Yang Liu, Bo Li
    AAAI Conference on Artificial Intelligence (AAAI), 2020 (Spotlight)
    [paper] [code]

    Multiple Character Embeddings for Chinese Word Segmentation
    Jingkang Wang*, Jianing Zhou*, Jie Zhou, Gongshen Liu
    Annual Meeting of the Association for Computational Linguistics (ACL), Student Research Workshop, 2019
    [paper] [code]

    LiDAR-Video Driving Dataset: Learning Driving Policies Effectively
    Yiping Chen*, Jingkang Wang*, Jonathan Li, Cewu Lu, Zhipeng Luo, Han Xue, Cheng Wang
    Conference on Computer Vision and Pattern Recognition (CVPR), 2018
    [paper] [code] [dataset]

    LightSim: Neural Lighting Simulation for Urban Scenes
    Ava Pun*, Gary Sun*, Jingkang Wang*, Yun Chen, Ze Yang, Sivabalan Manivasagam, Wei-Chiu Ma, Raquel Urtasun
    Advances in Neural Information Processing Systems (NeurIPS), 2023
    [paper][video][website]

    UniSim: A Neural Closed-Loop Sensor Simulator
    Ze Yang*, Yun Chen*, Jingkang Wang*, Sivabalan Manivasagam*, Wei-Chiu Ma,
    Anqi Joyce Yang, Raquel Urtasun
    Conference on Computer Vision and Pattern Recognition (CVPR), 2023 (Highlight)
    [paper] [video] [poster] [4K demo] [website]

    CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Simulation
    Jingkang Wang, Sivabalan Manivasagam, Yun Chen, Ze Yang, Ioan Andrei Bârsan, Anqi Joyce Yang,
    Wei-Chiu Ma, Raquel Urtasun
    Conference on Robot Learning (CoRL), 2022
    [paper] [video] [website]

    AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
    Jingkang Wang, Ava Pun, James Tu, Sivabalan Manivasagam, Abbas Sadat, Sergio Casas,
    Mengye Ren, Raquel Urtasun
    Conference on Computer Vision and Pattern Recognition (CVPR), 2021
    [paper] [poster] [video] [website]

    Adversarial Attack Generation Empowered by Min-Max Optimization
    Jingkang Wang*, Tianyun Zhang*, Sijia Liu, Pin-Yu Chen, Jiacen Xu, Makan Fardad, Bo Li
    Advances in Neural Information Processing Systems (NeurIPS), 2021
    [paper] [code] [poster] [website]

    Reinforcement Learning with Perturbed Rewards
    Jingkang Wang, Yang Liu, Bo Li
    AAAI Conference on Artificial Intelligence (AAAI), 2020 (Spotlight)
    [paper] [code]

    Education

    University of Toronto, Canada
    Ph.D. in Computer Science

  • Sept 2019 to Present
  • Advisor: Professor Raquel Urtasun

    • Shanghai Jiao Tong University, China
    B.S. in Engineering

  • Sept 2015 to June 2019

    • Experience

    Waabi Innovation Inc.
    Senior Researcher

  • Advisor: Professor Raquel Urtasun
  • Focus: Sensor Simulation, Self-Driving Safety
    		•

    Uber-ATG Toronto
    Research Scientist

  • Advisor: Professor Raquel Urtasun
  • Focus: Sensor Simulation, Self-Driving Safety
    		•   

    Ant Financial, Alibaba Group
    Research Intern

  • Advisor: Professor Le Song
  • Focus: Trustworthy Machine Learning

    • University of Illinois at Urbana-Champaign
    Research Intern at Secure Learning Lab   (remote work)

  • Advisor: Professor Bo Li
  • Focus: Robust Reinforcement Learning

    • University of California, Berkeley
    Research Intern at BAIR Lab   (remote work)

  • Advisor: Professors Bo Li and Dawn Song
  • Focus: Trustworthy Machine Learning

    • Machine Vision and Intelligence Group (MVIG)
    Research Assistant

  • Advisor: Professor Cewu Lu
  • Focus: Self Driving
    • Honors & Awards

  • Baidu Fellowship Finalist (Top 20)
    • 2021
  • Outstanding Reviewer, CVPR21
    • 2021
  • National Scholarships
    • 2016, 2017, 2018
  • Level-A SJTU Outstanding Scholarships
    • 2016, 2017, 2018
  • Excellent Bachelor Thesis (Top %1) of SJTU
    • 2019
  • Outstanding Undergraduate in Shanghai
    • 2019
  • First Prize in National College Student Information Security Contest
    • 2018
  • Meritorious Winner Prize in The Mathematical Contest in Modeling (MCM)
    • 2018
  • Second Prize in The Chinese Mathematics Competition (CMC, Shanghai)
    • 2017
  • Second Prize in National College Students Information Security Contest
    • 2017
  • First Prize in Chinese Mathematical Olympiad (CMO, 10th in Shanxi)
    • 2014

    Talks & Presentations

  • Safety-Critical Scenario Generation for Self Driving Vehicles. CVPR21 Tutorial: All about Self Driving. 2021/06. [video]
  • On the Importance of Initialization and Momentum in Deep Learning. CSC2541 Neural Net Training Dynamics. 2021/03. [slides]
  • Physics-based Differentiable Rendering. Reading group. 2021/03. [slides]
  • Differentiable Monte Carlo Ray Tracing through Edge Sampling. CSC2547 3D & Geometric Deep Learnig. 2021/02. [slides] [video][pptx]
  • Trust Region Policy Optimization (TRPO). CSC2621 Topics in Robotics. 2020/02. [slides]
  • Efficient Nonmyopic Active Search. CSC2547 Learning to Search. 2019/10. [slides]
  • Towards Secure and Interpretable Learning in Deep Neural Networks. Uber ATG. 2019/07. [slides]

    • Service

  • Conference reviewer: ICLR 2024, NeurIPS 2022-2023, ICCV 2021-2023, CVPR 2021-2023, ECCV 2022, WACV 2024, CoRL 2022, ICRA 2023, IROS 2023, AutoML 2023, ACL 2021-2022, NAACL 2022, EMNLP 2021, KDD 2020
  • Journal reviewer: IEEE TIP, IEEE TNNLS, CVIU, IEEE TSP, RA-L



    • Last update: June, 2023.

    Thanks jonbarron for this amazing work.