Overview

What you will learn

This course provides the basics behind computational approaches to working with images, starting with image acquisition (i.e. cameras/sensors), proceeding through understanding different representations (bitmap/raster, spectral), image manipulations (in-filling, mosaicing, etc.).

Who should take this course?

Anyone with an interest in pursuing computer vision (CSC420, CSC2503), deep learning (CSC421), computer graphics (CSC418) or computational photography (CSC2530).

Course Information Sheet

Please refer to the course information sheet for details of the course.

Announcements

Course announcements can be found on the course Piazza site.

Lectures/Schedule

Lecture slides will be available before each week's lecture. Demos and PDFs of research papers can be found in the Resources section.

Note: The lectures/slides are largely based on those of Kyros Kutulakos and Michael Guerzhoy.

(The schedule below is preliminary, and subject to change)

Part I: Representing images as 2D arrays of pixels

Week/Date Slides Topics Readings Other
1 (Jan. 9)

Topic 0: Introduction

Topic 1: Cameras

Introduction; Cameras and Images: Understanding digital images; basic camera controls; color image acquisition; image noise Sections 1.1-1.2, 2.1, 2.2, 2.4.2 (only paragraph entitled "silicon sensors"), 2.6.2 from Castleman book clarkvision.com: A very comprehensive website about photography, cameras and how to characterize their properties
2 (Jan. 16)

Topic 2: Matting

Topic 2.5: (Background) Solving Linear Systems of Equations

Topic 3: HDR Photography

HDR Imaging and Alpha Matting: Computing camera response functions from images; the matting equation Sections 1 and 2, up to Eq (2), from the Debevec 1997 Siggraph paper in Resources/Readings. As you read the paper, note that film response curve and camera response curve in the case of digital cameras, are one and the same. The HDRShop home page
Rendering with Natural Light (a movie that uses high-dynamic-range photography to capture outtdoor illumination and re-use it for image synthesis)

Part II: Representing images as continuous functions

Week/Date Slides Topics Readings Other
3 (Jan. 23) Computing 1D image derivatives: Least-squares polynomial fitting, intensity derivatives, weighted least squares, RANSAC To run the demo shown in class: (1) unpack the file polydemo.zip in the Demo Code directory, (2) run MATLAB, (3) change the current MATLAB directory to the directory you unpacked the code, (4) type polydemo at the matlab prompt. You should run the demo for a variety of fits (LS, WLS, 1st degree, 2nd degree, etc) to see their effect.
4 (Jan. 30) Edge detection: Local analysis of 1D and 2D image patches: the Image Gradient; case study: Painterly Rendering See Litwinowicz paper on painterly rendering in Resources/Readings (this is not required reading).
5 (Feb. 6) Corner detection, Intelligent Scissors, Seam Carving: Relation between local shape near extrema and the eigenvectors/eigenvalues of the Hessian; relation between eigenvalues & the trace & determinant of a matrix; localizing edges as zero crossings of the Laplacian; the Lowe feature detector: finding non-cylindrical points through eigenvalue analysis of the Hessian of the Laplacian See Mortensen paper on intelligent scissors in Resources/Readings (this is not required reading).
This paper and the technique will be covered in one of the forthcoming tutorials.

Part III: Representing images as N-dimensional vectors

Week/Date Slides Topics Readings Other
6 (Feb. 13) Template matching, correlation and patch-based image processing: Representing images as vectors; evaluating similarity using RMS distance error, cross-correlation and normalized cross-correlation; non-local means denoising To run the demo shown in class: (1) unpack the file corrdemo.zip in the Demo Code directory, (2) run MATLAB, (3) change the current MATLAB directory to the directory you unpacked the code, (4) type corrdemo at the matlab prompt.
7 (Feb. 27) Principal Component Analysis Section 13.6 from Castleman To run the demo shown in class: (1) unpack the file recognition_demo.zip in the Demo Code directory, (2) run MATLAB, (3) change the current MATLAB directory to the directory you unpacked the code, (4) type pca_recdemo at the matlab prompt.

Part IV: Multi-resolution image representations

Week/Date Slides Topics Readings Other
8 (Mar. 6) Gaussian Pyramids Original paper by Burt and Adelson on the Gauss/Laplacian pyramids in Sources/Readings. You should read up to, but not including, section entitled Entropy. The Matlab demo scale_space1D_demo.zip in the Demo Code directory on Dropbox shows how a 1D image changes as we smooth it with a sequence of Gaussians of increasing standard deviation.
9 (Mar 13) The Haar Wavelet Transform: Wavelet compression of 1D and 2D images The paper on the Haar Wavelets by Stollnitz et al in Resources/Readings. The Matlab demo wavedemo.zip in the Demo Code/ directory is the demo shown in class. Type wavedemo at the Matlab prompt to run it.
10 (Mar 20) Polynomial fitting vs. correlation; Matching images using SIFT: Analysis of WLS polynomial fitting and image smoothing as a template matching operation; template matching expressed as a multiplication of an image with a Toeplitz matrix; Gaussian image smoothing; SIFT-based feature detection; the SIFT descriptor; image matching using SIFT Sections 1-3 of the Lowe paper on SIFT found in the Readings/ directory on Dropbox. Web page on SIFT (with demo code)

Part V: Introduction to 2D Image Transformations

Week/Date Slides Topics Readings Other
11 (Mar. 27) Homogeneous coordinates: Homography-based image warping

Part VI: Learning Image Features

Week/Date Slides Topics Readings Other
12 (Apr. 3) Learning Image Filters with Neural Networks: Convolutional Neural Networks

Assignments

Submission and grading will be on MarkUs

Assignment # Due Date Handout Resources
1 Due: 12pm (noon), Jan. 30, 2018 A1 Handout A1 Starter Code
Smith and Blinn-SIGGRAPH 1996.pdf

Timetable

Lecture/Office Hours

Section Lecture Office Hours Instructor
LEC101 Wednesday 14:00 — 16:00, GB 119 Wednesday 16:00 — 17:00 , BA 2283 Yani Ioannou
LEC2501 / LEC5101 Wednesday 18:00 — 20:00, BA 1190 Monday 18:00 — 19:00, BA 2283 Yawen Ma
Both office hours are open to students from any section.

Tutorials

Section Time Location
TUT0101A/B Friday 14:00 — 15:00 GB 119
TUT5101A/B Wednesday 20:00 — 21:00 BA 1190

Resources

Listed here are a selection of unofficial resources that you may find helpful for this course.

Books

  • Computer Vision: Algorithms and Applications, Richard Szeliski (available online)
  • K. R. Castleman, Digital Image Processing, Prentice Hall, 1996
  • J. Minichino, J. Howse, Learning OpenCV3 Computer Vision with Python, 2nd ed, 2015

Readings

The readings are all available here

Websites

Contact

For help with assignments and course content, it is best to reach out to TAs and instructors in tutorials and office hours. TAs have a limited number of paid hours, and we would rather they spend that on in-person help and feedback. You can also find help from your fellow students and on the Piazza course forum


To reach out to the instructors/TAs about other matters, please use Piazza course forum to send a private post to "instructors", this will send a private message to the instructors.