Course Description

From the advanced depth cameras in today's smartphones to the 3D sensors of next-gen autonomous vehicles, 3D sensing has already began to revolutionize the way we sense the world. This, however, is just the beginning: cameras that can see in pitch-black darkness and around corners, depth sensors with sub-micron accuracy or a range of several kilometers, and cameras that can see under the skin are already becoming a reality. Key to these developments is the rapid convergence of optics, photonics, sensing and computation.

This seminar course will provide an in-depth look at 3D sensing and related technologies, and the computational techniques used to model and exploit them. Specific topics will include "extreme imaging" with single-photon detectors; principles of time-of-flight imaging and LiDAR; structured-light imaging; 3D imaging through scattering media; and imaging around corners. Class format will be discussion based, with approximately one paper covered per week and a term project.

Recommended preparation: Graduate-level exposure to one or more of computer vision, computer graphics, sensing or numerical optimization is desirable but not required. Prior enrollment in CSC2529 (Computational Imaging) is not necessary.

Teaching Assistant

Course Logistics

Lectures (starting Jan 14): Tuesdays 1:00pm-3:00pm in SS1073.

Instructor in-person office hour (starting Jan 22): Wednesdays 11am-noon (BA7270).
Instructor zoom office hour (starting Jan 14): Tuesdays 3:30-4:30pm (zoom link posted on Quercus).

Contact: Course announcements and general information will be posted Quercus. Q&A related to papers discussed in lecture will be on Piazza.

Course dropbox: The one-stop shop for all course materials. See Quercus for the link.

Role assignments & schedule: See Quercus for the link to the google sheet with all the info.

Coursework

Role playing participation (60%)

Marks will be distributed equally across all lectures (6% per lecture). All role-specific written components are due at midnight before the paper is discussed in class (ie. Mondays 11:59pm)

Term project (40%)

Project proposal is due February 25 (5% of mark). Final report is due at the end of classes (35% of mark).

Late policy

All homework is due at midnight on the due date. For role-specific homework, there will be a 30% marks deduction if you submit late, but before the start of that week's lecture (i.e., if you submit anytime between 12:01am and 1pm on Tuesday). No such homework will be accepted after the lecture's start since the homework's intent is to get your thoughts before any in-class discussions have occurred.

If you need more time to submit your project proposal or final project report, you will need to discuss your timeline with the instructor and get approval at least 1 week before the posted due date.

Lecture Schedule

Date Topic Paper(s) Event
Tue
14/1
Introduction
Course overview, role-playing class format, course components, grading, etc.
Part I: Active Sensing using Classical 3D Vision Cues
Tue
21/1
Laser
Triangulation
Main paper
Levoy et al., The digital Michelangelo project: 3D scanning of large statues (Proc. SIGGRAPH 2000).

Essential background
  • Curless, New methods for surface reconstruction from range images (PhD thesis). Chapters 1-2.

  • To probe further:
  • Curless, New methods for surface reconstruction from range images (PhD thesis). Chapters 3-4.
  • Curless et al., Better optical triangulation through spacetime analysis (Proc. ICCV 1995).
  • Tue
    28/1
    Structured-Light
    Triangulation
    Main paper
    Gupta & Nayar, Micro Phase Shifting (Proc. CVPR 2012)

    To probe further:
  • Nayar et al., Fast separation of direct and global components of a scene using high frequency illumination (Proc. SIGGRAPH 2006).
  • Szeliski & Scharstein, High-accuracy stereo depth maps using structured light (Proc. CVPR 2003).
  • Salvi et al., Pattern codification strategies in structured light systems (Pattern Recognition, 2004).
  • Zhang et al., Rapid shape acquisition using color structured light and multi-pass dynamic programming (Proc. 3DPVT 2002).
  • Tue
    4/2
    Photometric
    Stereo
    Main paper
    Johnson et al, Microgeometry capture using an elastomeric sensor (Proc. SIGGRAPH 2011).

    To probe further:
  • Hernandez et al., Non-rigid Photometric Stereo with Colored Lights (Proc. ICCV 2007).
  • Tue
    11/2
    Polarization
    Imaging
    Main paper
    Kadambi et al, Depth Sensing Using Geometrically Constrained Polarization Normals (IJCV 2017).

    To probe further:
  • Baek et al., Simultaneous acquisition of polarimetric SVBRDF and normals (SIGGRAPH Asia 2018).
  • Tue
    18/2
    Winter break (No Lecture)
    Part II: Time-Resolved 3D Sensing
    Tue
    25/2
    Single-Photon
    LiDAR
    Main paper
    Kirmani et al., First-Photon Imaging (Science 2014).
    Project proposals due at 11:59pm

    To probe further:
  • Gupta et al., Asynchronous Single-Photon 3D Imaging (Proc. ICCV 2019).
  • Tue
    4/3
    Indirect
    ToF
    Main paper
    Gupta et al., What Are Optimal Coding Functions for Time-of-Flight Imaging? (ACM TOG 2018).

    To probe further:
  • Lange & Seitz, Solid-state time-of-flight range camera (IEEE Trans. Quantum Electronics, 2001).
  • Achar et al., Epipolar Time of Flight (Proc. SIGGRAPH 2017).
  • Tue
    11/3
    Coherent
    ToF
    Main paper
    Li et al, Exploiting Wavelength Diversity for High Resolution Time-of-Flight 3D Imaging (IEEE PAMI 2017).

    To probe further:
  • Behroozpour et al., Lidar system architectures and circuits (IEEE Communications Mag. 2017).
  • Tue
    18/3
    Passive
    ToF
    Main paper
    Kotwal et al, Passive Micron-scale Time-of-Flight with Sunlight Interferometry (CVPR 2023).

    To probe further:
  • Gkioulekas et al, Micron-Scale Light Transport Decomposition Using Intererometry (SIGGRAPH 2015).
  • Tue
    25/3
    Coherent
    4D LiDAR
    Main paper
    Mirdehghan et al., Coherent Optical Modems for Full-Wavefield Lidar (ACM Siggraph Asia 2024).

    Tue
    1/4
    Non-Line-of-Sight 3D
    Imaging
    Main paper
    Velten et al, Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging (Nature Communications, 2012).

    To probe further:
  • Velten et al., Femto-photography: capturing and visualizing the propagation of light (SIGGRAPH 2013).
  • Tue
    25/4

    Final project reports due at 11:59pm

    Additional Information

    Related courses at UofT