CSC320
  • Description
  • Logistics
  • Courseware
  • Coursework
  • Schedule
  • Piazza

CSC320: Introduction to Visual Computing

Fall 2024

 

 

 

 

 

 

 

 

 

 

 

 

photo credit

Course Description

This course is a first-principles introduction to the acquisition and computational processing of 2D images. It is aimed at undergraduates interested in learning about computer vision, digital photography and computer graphics.

The course serves as a stepping stone for tackling more advanced courses in those subjects and covers four broad themes:

  • Mathematical and engineering foundations: introducing key concepts from geometry; multivariate calculus; linear algebra; image and signal processing; and human vision.
  • Algorithms for low-level computer vision: image warping, morphing and stitching; image enhancement; image scissoring and inpainting; color image processing and display; face recognition; and 2D image matching.
  • Implementation: implementing several such tools in Python and OpenCV.
  • A first taste of vision and graphics research: understanding how to turn algorithmic descriptions in research papers into working computer vision code---and how to evaluate its performance---will be key skills acquired in the course.

Instructor

Kyros Kutulakos

Teaching Assistants

Robin Swanson
Parsa Mirdehghan
Ali Saraer Toosi
Andrew Xie

Course Logistics

Section L0101
  Lectures: Tuesdays 1:00pm-3:00pm in BA1190.
  Tutorials: Thursdays 1:00pm-2:00pm in UC140.

Section L2501
  Lectures: Tuesdays 6:00pm-8:00pm in BA1190.
  Tutorials: Tuesdays 8:00pm-9:00pm in BA1190.

Instructor zoom office hour (week of Sept 10 onwards): Tuesdays 3:30-4:30pm (zoom link to be posted on Quercus)
Instructor in-person office hour (week of Sept 10 onwards): Wednesdays 11am-noon (BA7270).

TA zoom office hour: TBD (zoom link to be posted on Quercus).

Contact: Course announcements and general information will be posted Quercus. Q&A related to lectures, assignments and practice problems will take place on Piazza.

Course dropbox: The one-stop shop for all course materials. See Quercus for the link.

Course Materials

Lecture Slides & Recordings

Links to slides for all lectures will be available on the schedule below as well as on the course dropbox approximately an hour before each lecture. The full set of lecture slides from the most recent offering of the course (Spring 2024) is already available on dropbox. These slides will be very similar to slides for this term's offering so students who prefer reviewing the lecture content well in advance of a lecture can simply refer to those slides.

Recordings should be available on Quercus soon after each lecture. Since recording glitches are not uncommon due to issues with the system in specific classrooms (eg. missing sound, incomplete recordings, delays in posting) the completeness and timely availability of these recordings cannot be guaranteed, and they should not be treated as a substitute for in-person attendance.

Textbooks

In addition to lecture slides, assigned readings consist of selected sections from the following textbooks. The vast majority of required readings come from the first two; the rest will be used very occasionally and/or only as optional readings for those interested in delving more deeply into the lecture's subjects. Several of these books are freely available online; the rest can be found in the library and/or are available for purchase. See the course schedule for full details.

 

 

 

 

 

 

 

Videos

For the first half of the course, selected cartoon videos from Steve Seitz' "Graphics in 5 minutes (G5M)" channel on youtube must be watched before each week's lecture. In addition to these videos, Shree Nayar's video lectures on the First Principles of Computer Vision are an excellent source of lectures for optional viewing after class. See the course schedule for full details.

 

Problem Sets & Past Exams

Solutions to all past term tests and final exams since 2016 can be found on the course dropbox. The TAs have also compiled a selected set of practice problems from a variety of textbooks to test your knowledge of each lecture. See the course schedule for full details.

Coursework

Assignments (50%)

There will be 4 programming assignments in this class (see schedule). Each assignment comes with a partial implementation that you must then complete and test, and write a report about it. All except the first assignment will require reading an associated research paper and implementing (and testing) the technique it describes. Assignments will be exclusively in Python. All assignments will be posted on lecture day (Tuesdays) and will be due Thursday at midnight (11:59pm) three weeks later on MarkUs (except A4, which is due 20 days later). The three best assignments will count for 14% of the mark; the worst will count for 8%.

Late policy: All assignments are due at midnight on the due date. There will be a 15% marks deduction for each day late (i.e., if you submit your assignment anytime between 12:01am on the due date and 11:59pm the next day your penalty will be 15%). No assignments will be accepted more than 5 days late.

To account for unexpected technical last-minute issues with submissions, each student will be allocated one-hour grace period "budget". If submission of A1 or any other assignment is only a few minutes late, these minutes will be deducted from the grace period budget. Once that budget goes to zero, submitting a future assignment even a minute late will incur the full one-day lateness penalty. Assignments submitted more than 1 hour late are not eligible for grace periods and do not affect the grace period budget. If you wish to apply all or part of your budget for an assignment you submitted a few minutes late, you must submit the online CSC320 special consideration request form (link availabe on Quercus). No grace period will be given without such a submission.

Aside from this budget, requests for special consideration can only be made for health conditions, injury, personal or family emergency, or bereavement. All such requests must be made by submitting the online CSC320 special consideration request form (link availabe on Quercus). You must submit this form even if you have already made an absence declaration request on Acorn. Requests made over email to an instructor or TA will not be honored.

Academic honesty policy for written components: All reports submitted as part of your assignments are strictly individual work. No part of these reports should be shared with others, or taken from others. This includes verbatim text, paraphrased text, and/or images used. You are, however, allowed to discuss these components with others at the level of ideas, and indeed you are welcome to brainstorm together.

Academic honesty policy for programming components: Collaboration on a programming component by individuals (whether or not they are taking the class) is encouraged at the level of ideas. Feel free to ask each other questions, brainstorm on algorithms, or work together on a (virtual or real) whiteboard. Be careful, however, about copying actual code for programming assignments or merely adapting someone else's code. This sort of collaboration at the level of artifacts is permitted if explicitly acknowledged, but will result in marks deduction. Full details about how to acknowledge such collaborations (and what deductions to expect) will be provided in the individual assignment handouts.

The use of generative artificial intelligence (AI) tools is strictly prohibited in all programming assignments. This includes, but is not limited to, ChatGPT, GitHub Copilot, and open-source models that you have trained and/or deployed yourself. You may not interact with, nor copy, paraphrase, or adapt any content from any generative AI for the purpose of completing programming assignments in this course. Use of generative AI will be considered use of an unauthorized aid, which is a form of academic misconduct under the Code of Behavior on Academic Matters. This course policy is designed to promote your learning and intellectual development and to ensure that our evaluations are a fair and accurate assessment of your learning. To that end, you will be required to include detailed comments along with the code you submit, and a random subset of students may be selected by a TA to explain their implementation in a live 1-1 meeting.

Exams (50%)

Term test 1 (15%): Oct 15, 8-9pm (all sections). Rooms TBD.
Final exam (35%, no auto-fail): Date and time TBD.

Schedule and Syllabus

Lecture
Week
Date Description G5M Videos to Watch Before Class Lectures, Readings &
Exercises
Event Deadline
Part I: Foundations
1 Tue
3/9
Course Intro
Image Geometry 1
Homogeneous 2D coordinates, 2D transformations, homographies, image warping algorithms.
Affine transforms
[slides01][materials01]
Tue
3/9
Tutorial
Image I/O in OpenCV and Python.
[tslides01]
Thu
5/9
Tutorial
Image I/O in OpenCV and Python.
2 Tue
10/9
Image Geometry 2
Homogeneous 3D coordinates, pinhole imaging & perspective projection, imaging of 3D planes, panoramic image stitching, homography estimation.
Perspective projection,
Perspective projection--the math
[slides02] [materials02] A1 out on Dropbox
Tue
10/9
Tutorial
Linear algebra refresher: solving systems of linear equations. Lecture 1 practice problems.
[tslides02]
The
12/9
Tutorial
Linear algebra refresher: solving systems of linear equations. Lecture 1 practice problems.
3 Tue
17/9
Image Filtering 1
Linear shift-invariant filters, convolution, basic filters.
Images,
Image filtering
[slides03] [materials03]
Tue
17/9
Tutorial
Lecture 2 practice problems. A1 Q&A.
[tslides03]
Thu
19/9
Tutorial
Lecture 2 practice problems. A1 Q&A.
4 Tue
24/9
Image Filtering 2
Gaussian filters, derivative filters, sharpening filters. The Bilateral filter. Discrete image formation, image interpolation.
Interpolation [slides04] [materials04]
Tue
24/9
Tutorial
Image filtering in action with OpenCV. Lecture 3 practice problems. Lecture 6 prep: Math refresher on complex numbers and their Euler representation.
[tslides04]
Thu
26/9
Tutorial
Image filtering in action with OpenCV. Lecture 3 practice problems. Lecture 6 prep: Math refresher on complex numbers and their Euler representation.
5 Tue
1/10
Color Vision & Camera ISPs
Color perception in the human visual system, color imaging, color spaces, color displays. Smartphone Image Signal Processing pipelines.
Color [slides05] [materials05] A2 out on Dropbox
Tue
1/10
Tutorial
Lecture 3 & 4 practice problems. A2 overview and Q&A.
[tslides05]
Thu
3/10
Tutorial
Lecture 3 & 4 practice problems. A2 overview and Q&A.
A1 due at 11:59pm
6 Tue
8/10
Image Filtering 3
Introduction to Fourier transforms, convolution theorem.
Fourier transforms [slides06] [materials06]
Tue
8/10
Image Filtering 3 leture continued (no tutorial this week)
Mathematics of Fourier Transforms.
[tslides06]
Thu
10/10
Image Filtering 3 lecture continued (no tutorial this week)
Mathematics of Fourier Transforms.
7 Tue
15/10
Image Filtering 4
Fourier-domain image filtering, hybrid images. Sampling theory basics, aliasing, anti-aliasing methods.
[no video] [slides07] [materials07]
15/10 Term Test 1 (8-9pm, all sections, rooms TBD, covers Week 1-6 materials)
Thu
17/10
Tutorial
Fourier-domain filtering & anti-aliasing in action with OpenCV. A2 Q&A.
[tslides07]
8 Tue
22/10
Differentiable 1D & 2D Representations
Representing smooth 2D curves, curve tangent, curve normal and the moving frame. Image gradient, image Laplacian, edge enhancement, Canny edge detection. Intelligent scissors, painterly rendering.
[no video] [slides08] [materials08] A3 out on Dropbox
Tue
22/10
Tutorial
Lecture 7 practice problems. A3 Q&A.
[tslides08]
Thu
24/10
Tutorial
Lecture 7 practice problems. A3 Q&A.
A2 due at 11:59pm
29/10
31/10
Fall break (No Lecture)
9 Tue
5/11
Vector-Based Representations 1
Template matching, measuring image similarity, Harris/Foerstner corner detection. Introduction to PCA.
[no video] [slides09] [materials09]
Tue
5/11
Tutorial
Lecture 8 practice problems. Lecture 10 prep: Linear algebra refresher on matrix eigenvectors and eigenvalues.
Thu
7/11
Tutorial
Lecture 8 practice problems. Lecture 10 prep: Linear algebra refresher on matrix eigenvectors and eigenvalues.
10 Tue
12/11
Vector-Based Representations 2
Principal component analysis, face recognition using eigenfaces.
[no video] [slides10] [materials10] A4 out on Dropbox
Tue
12/11
Tutorial
Lecture 9 practice problems. Overview of A4.
[tslides10]
Thu
14/11
Tutorial
Lecture 9 practice problems. Overview of A4.
A3 due at 11:59pm
11 Tue
19/11
Multi-Scale Representations 1
The Discrete Haar Wavelet Transform, wavelet-based image compression, wavelet-based image processing.
[no video] [slides11] [materials11]
Tue
19/11
Tutorial
Lecture 10 practice problems. A4 Q&A.
[tslides11]
Thu
21/11
Tutorial
Lecture 10 practice problems. A4 Q&A.
12 Tue
26/11
Multi-Scale Representations 2
Panoramic image stitching revisited. SIFT keypoints and SIFT-based correspondence-finding. Robust model fiting using RANSAC. Gaussian and Laplacian pyramids. Laplacian image blending.
[no video] [slides12] [materials12]
Tue
26/11
Tutorial
Lecture 11 practice problems. A4 Q&A. Preparing for the final.
[tslides12]
Thu
28/11
Tutorial
Lecture 11 practice problems. A4 Q&A. Preparing for the final.
Mon
2/12
End of term
A4 due at 11:59pm

Additional Information

Prerequisites

  • Formal prerequisites: Please consult the Arts and Science timetable for full details on prerequisites and exclusions. Requests for prerequisite waivers should be directed to the Computer Science Undergraduate office.
  • Coding Experience: All assignments will be in Python and will rely heavily on packages such as NumPy and OpenCV. Advance knowledge of these packages is not necessary; they will be covered in tutorials as needed.
  • Course sequencing: Students planning to take CSC420 (Introduction to Image Understanding) are strongly encouraged to take CSC320 first. Both courses are currently in a transitional phase, with foundational material progressively shifted exclusively to CSC320 to make room for teaching more advanced topics in CSC420. Students planning to take CSC317 (Computer Graphics) would also greatly benefit from taking CSC320 first, or concurrently with that course.

Hardware & Software Compatibility

  • You are welcome to use your personal computer (OSX, Windows or Linux) for programming assignments. Be aware, however, that TAs will test your code on Teaching Labs computers and thus it is essential that you verify your code there before submission.

  • We will support only one of the latest versions of python and recommend that you install it using Anaconda or Miniconda (see installation instructions here). Further information about setting up your computing environment for the course's programming assignents will be given at the first tutorial and will also be available in the Tutorial 1 slides.

Related courses at UofT

  • CSC317: Computer Graphics
  • CSC420: Introduction to Image Understanding (UTM)
  • CSC420: Introduction to Image Understanding (StG)
  • CSC2503: Foundations of Computer Vision
  • CSC2529: Computational Imaging
  • CSC2530: Computational Imaging and 3D Sensing
  • CSC2520: Geometry Processing

Acknowledgements

The lecture slides in this installment of the course have been revised significantly from offerings of the course prior to 2021, and include slides from several other instructors. Feel free to use these slides for academic or research purposes, but please maintain all acknowledgments and source information. The format and style of this webpage are based on those of CS2529, taught by David Lindell.