Computational imaging systems have a wide range of applications in consumer electronics, scientific imaging, HCI, medical imaging, microscopy, and remote sensing. We discuss digital photography and basic image processing, convolutional neural networks for image processing, denoising, deconvolution, single pixel imaging, inverse problems in imaging, proximal gradient methods, introduction to wave optics, time-of-flight imaging, end-to-end optimization of optics and imaging processing. Emphasis is on applied image processing and solving inverse problems using classic algorithms, formal optimization, and modern artificial intelligence techniques. Students learn to apply material by implementing and investigating image processing algorithms in Python and completing a term project.
Topics include:
Lectures: Monday 10:00am-12:00pm in Galbraith Building 120. Lectures will be recorded and made available on Quercus.
Instructor office hours: Monday 1:30-2:30pm (BA7228) starting on September 12; discussion about projects, material, etc. TA office hours: Tuesdays and Fridays in Bahen 3201 from 12:00-1:30pm starting on September 16; discussion about homework questionsProblem sessions: Wednesday 11:00am-12:00pm (SS1071) starting on September 14. Problem sessions will be recorded and made available on Quercus.
Textbook: There is no textbook for the course; links to readings and course notes are provided in the schedule.
Contact: Course announcements and general information will be posted on the course forum at Ed Discussion. For any emergency inquiries that cannot be handled on the forum via a private post, email the staff listserv: csc2529-fall2223-staff-l@listserv.utoronto.ca.
All assignments and the final project proposal and report should be submitted on Gradescope using entry code MXDXRM. If you work as a team, make sure to indicate your team member in the submission.
There will be 6 homework assignments (see schedule) in this class. These assignments will contain some theoretical questions and also implementations of techniques that we will discuss in class. Please refer to assignment writeups (available in the Files tab on Quercus) for details. After you finish, submit your code and report on Gradescope.
There are no "late days" for the assignments. If you choose to submit an assignment late, we will accept it for up to 24h after the submission deadline with a 30% penalty (final grade multiplied by 0.7).
Collaboration Policy: Students are permitted to work together on homework assignments in small groups. However, while students can discuss the assignments together, they must compose their solutions individually. Students must not view or copy code of other students or solutions available on the internet. Names of collaborators must be listed on submitted assignments. Submitting plagiarized solutions is an academic offense and can have severe penalties.
The final project grade takes into account your poster presentation (organization of poster, clarify of presentation, ability to answer question), your source code submission (code organization and documentation), and your final project report (appropriate format and length, abstract, introduction, related work, description of your method, quantitative and qualitative evaluation of your method, results, discussion & conclusion, bibliography).
You can work in teams of up to 3 students for the project. Submit only one proposal and final report for each team. The expected amount of work is relative to the number of team members, so if two teams work on a similar project, we'd expect less work from a smaller team. Before you start to work on the proposal or the report, take a look at some of the past project proposals and reports to give you sense for what's expected (see link at the bottom of this page).
The project proposal is a 1-2 page document that should contain the following elements: clear motivation of your idea, a discussion of related work along at least 3 scientific references (i.e., scientific papers not blog articles or websites), an overview of what exactly your project is about and what the final goals are, milestones for your team with a timeline and intermediate goals. Once you send us your proposal, we may ask you to revise it and we will assign a project mentor to your team.
The final project report should look like a short (~6 pages) conference paper. We expect the following sections, which are standard practice for conference papers: abstract, introduction, related work, theory (i.e., your approach), analysis and evaluation, results, discussion and conclusion, references. To make your life easier, we provide an LaTex template that you can use to get started on your report (see schedule for link).
Week | Date | Description | Material | Readings | Event | Deadline |
---|---|---|---|---|---|---|
Week 1 | Mon 12/9 |
Lecture 1: Course intro, human visual system Overview of class, human perception of color, depth, contrast, resolution |
[slides]
|
Hybrid Images Paper | HW1 out on Quercus | |
Wed 14/9 |
Problem session (HW1) | [slides] | ||||
Fri 16/9 |
TA Office Hours (HW1) | |||||
Week 2 | Mon 19/9 |
Lecture 2: Digital photography I Ray optics, aperture, depth of field, exposure, sensor, noise |
[slides] | Marc Levoy's course on digital photography | HW2 out on Quercus | |
Tue 20/9 |
TA Office Hours (HW1) | |||||
Wed 21/9 |
HW1 due at 11:59pm | |||||
Week 3 | Mon 26/9 |
Lecture 3: Digital photography II CameraISP, demosaicking, denoising, deconvolution |
[slides] |
Demosaicking Paper Non-local Means Paper Intro to Bilateral Filtering |
||
Tue 27/9 |
TA Office Hours (HW2) | |||||
Wed 28/9 |
Problem session (HW2) | [slides] | ||||
Fri 30/9 |
TA Office Hours (HW2) | |||||
Week 4 | Mon 3/10 |
Lecture 4: Math review Quick review of sampling, optimization, deconvolution, ... |
[slides] | HW3 out on Quercus | ||
Wed 5/10 |
Problem session (HW3) | [slides] | HW2 due at 11:59pm | |||
Fri 7/10 |
TA Office Hours (HW3) | |||||
Week 5 | Mon 10/10 |
Thanksgiving Day (No Lecture) | ||||
Week 6 | Mon 17/10 |
Lecture 5: Great ideas in computational photography HDR, tone mapping, coded apertures, flutter shutter |
[slides] |
HDR Imaging Paper Tone Mapping Paper Ext. Depth of Field Paper Flutter Shutter Paper Learned Coded Apertures Neural Sensors |
HW4 out on Quercus | |
Tue 18/10 |
TA Office Hours (HW3) | |||||
Wed 19/10 |
Problem session (HW4) | [slides] | HW3 due at 11:59pm | |||
Fri 21/10 |
TA Office Hours (HW4) | |||||
Week 7 | Mon 24/10 |
Lecture 6: Introduction to neural networks MLPs, CNNs, ResNets, denoising/deconvolution with CNNs |
[slides] |
Linear classifiers
Backpropagation notes
Optimization notes |
HW5 out on Quercus | |
Tue 25/10 |
TA Office Hours (HW4) | |||||
Wed 26/10 |
Problem session (HW5) | [slides] | HW4 due at 11:59pm | |||
Fri 28/10 |
TA Office Hours (HW5) | |||||
Week 8 | Mon 31/10 |
Lecture 7: Solving regularized inverse problems with ADMM Natural image priors, deconvolution, single-pixel imaging, ADMM, solving general inverse problems |
[slides] |
Deconvolution notes
Compressive imaging notes |
HW6 out on Quercus | |
Tue 1/11 |
TA Office Hours (HW5) | |||||
Wed 2/11 |
Problem session (HW6) | [slides] | HW5 due at 11:59pm | |||
Fri 4/11 |
TA Office Hours (HW6) | |||||
Week 9 | Mon 7/11 |
Fall Reading Week (No Lecture) | ||||
Week 10 | Mon 14/11 |
Lecture 8: Light Field Imaging plenoptic function, light field cameras, 3D displays |
[slides] | |||
Tues 15/11 |
TA Office Hours (HW6) | |||||
Wed 16/11 |
HW6, Project Proposal due at 11:59pm | |||||
Week 11 | Mon 21/11 |
Lecture 9: Time-of-flight imaging lidar, single-photon imaging, non-line-of-sight imaging, scattering |
[slides] | |||
Week 12 | Mon 28/11 |
Lecture 10: Neural Field Representations neural fields, NeRF, applications |
[slides] | |||
Week 13 | Mon 5/12 |
Lecture 11: Guest Lecture Mark Sheinin (CMU) Computational Imaging for Enabling Vision Beyond Human Perception |
[slides] | |||
Thu 8/12 |
Final Project - poster presentations in Bahen lobby |
ProjectPosterTemplate.ppt iccp2020_latex_template.zip |
Project report and code due at 11:59pm |
We will only support Python 3.7 and recommend that you install it using Anaconda or Miniconda (see installation instructions here).
See Quercus for instructions on how to print your poster for the final project poster session.
This course is adapted from the Computational Imaging course designed by Gordon Wetzstein and offered at Stanford University (EE367). Below you can find links to pinhole camera photos and course projects from these previous iterations of the course.
Links to notable pinhole camera photos:
The course is adapted from EE367 at Stanford University by Gordon Wetzstein. Some of the materials used in class build on that from other instructors, including Yannis Gkioulekas, Marc Levoy, Fredo Durand, Ramesh Raskar, Shree Nayar, Paul Debevec, Matthew O'Toole and others, as noted in the slides. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgments. This webpage is based on the website for CS231N and EE367 at Stanford University.