From the advanced depth cameras in today's smartphones to the 3D sensors of next-gen autonomous vehicles, 3D sensing has already began to revolutionize the way we sense the world. This, however, is just the beginning: cameras that can see in pitch-black darkness and around corners, depth sensors with sub-micron accuracy or a range of several kilometers, and cameras that can see under the skin are already becoming a reality. Key to these developments is the rapid convergence of optics, photonics, sensing and computation.
This seminar course will provide an in-depth look at 3D sensing and related technologies, and the computational techniques used to model and exploit them. Specific topics will include "extreme imaging" with single-photon detectors; principles of time-of-flight imaging and LiDAR; structured-light imaging; 3D imaging through scattering media; and imaging around corners. Class format will be discussion based, with approximately one paper covered per week and a term project.
Recommended preparation: Graduate-level exposure to one or more of computer vision, computer graphics, sensing or numerical optimization is desirable but not required. Prior enrollment in CSC2529 (Computational Imaging) is not necessary.
Lectures (starting Jan 14): Tuesdays 1:00pm-3:00pm in SS1073.
Instructor in-person office hour (starting Jan 22): Wednesdays 11am-noon (BA7270).
Instructor zoom office hour (starting Jan 14): Tuesdays 3:30-4:30pm (zoom link posted on Quercus).
Contact: Course announcements and general information will be posted Quercus. Q&A related to papers discussed in lecture will be on Piazza.
Course dropbox: The one-stop shop for all course materials. See Quercus for the link.
Role assignments & schedule: See Quercus for the link to the google sheet with all the info.
Marks will be distributed equally across all lectures (6% per lecture). All role-specific written components are due at midnight before the paper is discussed in class (ie. Mondays 11:59pm)
Project proposal is due February 25 (5% of mark). Final report is due at the end of classes (35% of mark).
All homework is due at midnight on the due date. For role-specific homework, there will be a 30% marks deduction if you submit late, but before the start of that week's lecture (i.e., if you submit anytime between 12:01am and 1pm on Tuesday). No such homework will be accepted after the lecture's start since the homework's intent is to get your thoughts before any in-class discussions have occurred.
If you need more time to submit your project proposal or final project report, you will need to discuss your timeline with the instructor and get approval at least 1 week before the posted due date.
Date | Topic | Paper(s) | Event |
---|---|---|---|
Tue 14/1 |
Introduction |
Course overview, role-playing class format, course components, grading, etc. | |
Part I: Active Sensing using Classical 3D Vision Cues | |||
Tue 21/1 |
Laser Triangulation |
Main paper Levoy et al., The digital Michelangelo project: 3D scanning of large statues (Proc. SIGGRAPH 2000). |
|
Essential background
To probe further: |
|||
Tue 28/1 |
Structured-Light Triangulation |
Main paper Gupta & Nayar, Micro Phase Shifting (Proc. CVPR 2012) |
|
To probe further:
|
|||
Tue 4/2 |
Photometric Stereo |
Main paper Johnson et al, Microgeometry capture using an elastomeric sensor (Proc. SIGGRAPH 2011). |
|
To probe further:
|
|||
Tue 11/2 |
Polarization Imaging |
Main paper Kadambi et al, Depth Sensing Using Geometrically Constrained Polarization Normals (IJCV 2017). |
|
To probe further:
|
|||
Tue 18/2 |
Winter break (No Lecture) | ||
Part II: Time-Resolved 3D Sensing | Tue 25/2 |
Single-Photon LiDAR |
Main paper Kirmani et al., First-Photon Imaging (Science 2014). |
Project proposals due at 11:59pm |
To probe further:
|
|||
Tue 4/3 |
Indirect ToF |
Main paper Gupta et al., What Are Optimal Coding Functions for Time-of-Flight Imaging? (ACM TOG 2018). |
|
To probe further: | |||
Tue 11/3 |
Coherent ToF |
Main paper Li et al, Exploiting Wavelength Diversity for High Resolution Time-of-Flight 3D Imaging (IEEE PAMI 2017). |
|
To probe further: | |||
Tue 18/3 |
Passive ToF |
Main paper Kotwal et al, Passive Micron-scale Time-of-Flight with Sunlight Interferometry (CVPR 2023). |
|
To probe further: | |||
Tue 25/3 |
Coherent 4D LiDAR |
Main paper Mirdehghan et al., Coherent Optical Modems for Full-Wavefield Lidar (ACM Siggraph Asia 2024). |
|
Tue 1/4 |
Non-Line-of-Sight 3D Imaging |
Main paper Velten et al, Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging (Nature Communications, 2012). |
|
To probe further: | Tue 25/4 |
|
Final project reports due at 11:59pm |