Overview

What you will learn

This course provides the basics behind computational approaches to working with images, starting with image acquisition (i.e. cameras/sensors), proceeding through understanding different representations (bitmap/raster, spectral), image manipulations (in-filling, mosaicing, etc.).

Who should take this course?

Anyone with an interest in pursuing computer vision (CSC420, CSC2503), deep learning (CSC421), computer graphics (CSC418) or computational photography (CSC2530).

Course Information Sheet

Please refer to the course information sheet for details of the course.

Announcements

Course announcements can be found on the course Piazza site.

Lectures/Schedule

Lecture slides will be available before each week's lecture. Demos and PDFs of research papers can be found in the Resources section.

Note: The lectures/slides are largely based on those of Kyros Kutulakos and Michael Guerzhoy.

(The schedule below is preliminary, and subject to change)

Part I: Representing images as 2D arrays of pixels

Week/Date Slides Topics Readings Other
1 (Jan. 9)

Topic 1: Cameras

 Introduction; Cameras and Images: Understanding digital images; basic camera controls; color image acquisition; image noise Sections 1.1-1.2, 2.1, 2.2, 2.4.2 (only paragraph entitled "silicon sensors"), 2.6.2 from Castleman book clarkvision.com: A very comprehensive website about photography, cameras and how to characterize their properties
2 (Jan. 16)

Topic 2: Matting

Topic 2.5: (Background) Solving Linear Systems of Equations

Topic 3: HDR Photography

 HDR Imaging and Alpha Matting: Computing camera response functions from images; the matting equation Sections 1 and 2, up to Eq (2), from the Debevec 1997 Siggraph paper in Resources/Readings. As you read the paper, note that film response curve and camera response curve in the case of digital cameras, are one and the same. The HDRShop home page
Rendering with Natural Light (a movie that uses high-dynamic-range photography to capture outtdoor illumination and re-use it for image synthesis)

Part II: Representing images as continuous functions

Week/Date Slides Topics Readings Other
3 (Jan. 23 - Yawen Ma) Topic 4: Local Analysis of Image Patches (1D) Computing 1D image derivatives: Least-squares polynomial fitting, intensity derivatives, weighted least squares, RANSAC To run the demo shown in class: (1) unpack the file polydemo.zip in the Demo Code directory, (2) run MATLAB, (3) change the current MATLAB directory to the directory you unpacked the code, (4) type polydemo at the matlab prompt. You should run the demo for a variety of fits (LS, WLS, 1st degree, 2nd degree, etc) to see their effect.
4 (Jan. 30 - Yani Ioannou) Topic 5: 2D Image Patches and Curves Edge detection: Local analysis of 1D and 2D image patches, the image gradient, edge detection, case study: RANSAC Circle finding Code for this week's lecture.

Part III: Representing images as N-dimensional vectors

Week/Date Slides Topics Readings Other
5 (Feb. 6 - Yani Ioannou) Cancelled due to University Closure - rescheduled to week 7. Topic 6: Images as Vectors Template matching, correlation and patch-based image processing: Representing images as vectors; evaluating similarity using RMS distance error, cross-correlation and normalized cross-correlation; dimensionality reduction; principle component analysis; case study: face recognition using Eigenfaces Section 13.6 from Castleman, see many links in slides on understanding PCA
6 (Feb. 13 - Yawen Ma) Topic 7: Convolution Convolution; Analysis of WLS polynomial fitting and image smoothing as a template matching operation; Template matching expressed as a multiplication of an image with a Toeplitz matrix; Gaussian image smoothing; Interpolation
7 (Feb. 27 - Midterm) Midterm Exam
(NOT in class)
Time: 20:10-21:00
Location: EX 200 (Exam Centre)		 Topics covered: Everything up to and including week 6 (excluding the cancelled lecture of week 5) Note: This is outside of the scheduled lecture time/room! Please check for any conflicts and let the instructors know ASAP.
7 (Feb. 27 - Yani Ioannou) Topic 6: Images as Vectors (Rescheduled from Week 5) Template matching, correlation and patch-based image processing: Representing images as vectors; evaluating similarity using RMS distance error, cross-correlation and normalized cross-correlation; dimensionality reduction; principle component analysis; case study: face recognition using Eigenfaces Section 13.6 from Castleman, see many links in slides on understanding PCA
8 (Mar. 6 - Yani Ioannou) Topic 8: Images in the Frequency Domain Fourier Transform, Convolution Theorem Code for this week's lecture

Part IV: Introduction to 2D Image Transformations

Week/Date Slides Topics Readings Other
9 (Mar 13 - Yawen Ma) Topic 9: Wavelets The Haar Wavelet Transform: Wavelet compression of 1D and 2D images; Intelligent Scissors The paper on the Haar Wavelets by Stollnitz et al in Resources/Readings. See Mortensen paper on intelligent scissors in Resources/Readings (this is not required reading).
10 (Mar. 20 - Yani Ioannou) Topic 10: Image Pyramids
Topic 11: SIFT		 Image Pyramids: Gaussian Pyramids, Laplacian Pyramids. Matching images using SIFT;SIFT-based feature detection; the SIFT descriptor; image matching using SIFT Original paper by Burt and Adelson on the Gauss/Laplacian pyramids in Sources/Readings. You should read up to, but not including, section entitled Entropy. Sections 1-3 of the Lowe paper on SIFT found in the Readings. Web page on SIFT (with demo code)
SIFT Paper
OpenCV SIFT Description
11 (Mar. 27 - Yawen Ma) Topic 12: Morphing
Topic 13: Homographies and Image Mosaics		 Homogeneous coordinates: Homography-based image warping; Homographies; Image morphing: backward mapping

Part V: Learning Image Features

Week/Date Slides Topics Readings Other
12 (Apr. 3 - Yani Ioannou) Topic 14: Learning to Understand Images Learning Image Filters with Neural Networks: Convolutional Neural Networks

Assignments

Submission and grading will be on MarkUS

Assignment # Due Date Handout Resources
1 Due: 12pm (noon), Jan. 30, 2019 A1 Handout A1 Starter Code
Smith and Blinn-SIGGRAPH 1996.pdf
2 Due: 12pm (noon), Mar. 1, 2019 A2 Handout A2 Starter Code
A2 CDF Reference Binary
Criminisi-IEEE-TIP 2004.pdf
3 Due: 12pm (noon), Mar. 20, 2019 A3 Handout A3 Starter Code
Barnes_Siggraph2009.pdf
4 Due: 12pm (noon), Apr. 4, 2019 A4 Handout A4 Starter Code
A4 Reference Results
Barnes_ECCV_2010.pdf
Buades_CVPR2005.pdf

Timetable

Lecture/Office Hours

Section Lecture Office Hours Instructor
LEC0101 Wednesday 14:00 — 16:00, GB 119 Wednesday 16:00 — 17:00 , BA 2283 Yani Ioannou / Yawen Ma
LEC2501 / LEC5101 Wednesday 18:00 — 20:00, BA 1190 Monday 18:00 — 19:00, BA 2283 Yani Ioannou / Yawen Ma
Both office hours are open to students from any section.

Tutorials

Section Time Location
TUT0101A/B Friday 14:00 — 15:00 GB 119
TUT5101A/B Wednesday 20:00 — 21:00 BA 1190

Resources

Listed here are a selection of unofficial resources that you may find helpful for this course.

Books

  • Computer Vision: Algorithms and Applications, Richard Szeliski (available online)
  • K. R. Castleman, Digital Image Processing, Prentice Hall, 1996
  • J. Minichino, J. Howse, Learning OpenCV3 Computer Vision with Python, 2nd ed, 2015

Readings

The readings are all available here

Websites

Contact

For help with assignments and course content, it is best to reach out to TAs and instructors in tutorials and office hours. TAs have a limited number of paid hours, and we would rather they spend that on in-person help and feedback. You can also find help from your fellow students and on the Piazza course forum


To reach out to the instructors/TAs about other matters, please use Piazza course forum to send a private post to "instructors", this will send a private message to the instructors.