CSC2125: Software Engineering for ML-Enabled Systems (SE4AI)

Instructor: Marsha Chechik, chechik@cs.toronto.edu

TA: Caroline Hu, bouye@cs.toronto.edu

Lectures: Mondays 12-3 (bring a snack or drink!) in OI 8170 (OISE, 252 Bloor Street W.)

Office Hours: Mondays 3:30-4:30 or by appointment, Bahen Centre (40 St. George Street, Rm. 3259)

Goals of the course:

Understand how ML components are parts of larger systems

Understand the challenges in engineering an ML-enabled system beyond accuracy

Understand the role of specifications in machine learning and the relationship to deductive and inductive reasoning

Study the challenges and techniques of verification and validation of such systems

Discuss safety, reliability and (optionally) other AI-related issues such as fairness

Prerequisites:

No knowledge of SE is expected:

requirements, software testing, software design, process

(but of course always helpful)

Assume SOME knowledge of ML:

Basic understanding of data science process, including data cleaning, feature engineering, using ML libraries

High level understand of machine-learning approaches

supervised learning

regression, decision trees, neural networks

accuracy, recall, precision

Tentative Syllabus:

• Introduction: SE challenges in systems with AI components

• Requirements and risks

• ML process and deployment

• Testing (unit and system), intro to V&V

• Safety and reliability

• Data management

• Advanced topics, if time allows: explainability, ethics, fairness

• Whatever interests you! (based on selection of papers to present)

Course Calendar

1. Sept. 11 Lecture 1: intro Assignment 1 out

2. Sept. 18 Lecture 2: requirements Due: Assignment 0

3. Sept. 25 Due: expressions of paper interest

4. Oct. 2

Oct. 9 Thanksgiving. No class.

5. Oct. 16 Due: Research topic proposal,

Assignment 1. Assignment 2 out

6. Oct. 23

7. Oct. 30

8. Nov. 6 Fall reading week. Class on. Due: Assignment 2. Assignment 3 out

9. Nov. 13

10. Nov. 20

11. Nov. 27

Dec. 4 No class. Due: Assignment 3, Project

Dec. 18? Project presentations

Course Workload – two flavors

“Hands-on”

70% 3 assignments, done in groups of 2

(A1 is 25%, A2 is 15%, A3 is 30%)

15% Presentation of one paper (individual, starting from week 3)

15% Course participation

OR

“Research”

50% Research project (individual or group)

35% Presentation of two papers (individual), at least one of them “more technical”

15% Course participation

Reading List

Book: Building Intelligent Systems by Geoff Hulten. Available from UofT Library here

List of papers for presentations is available HERE

No color – somewhat easier papers

Red – somewhat harder

Yellow – talk to us if you want to present

Information about the Assignments

3 small to medium-sized assignments done with a partner with the goal of engaging with practical challenges, reason about tradeoffs and justify your decisions, write reports, do some modeling and some coding. Possibly present results during last class.

Information about the Project

Design your own research project and write a report

- A case study, empirical study, literature survey, etc.

- Can be done individually or in a group of 2

- Project proposals due October 16

- Very open ended: Align with own research interests and existing projects

Information about Paper Presentations

Academic honesty

In a nutshell: do not copy from other students, do not lie, do not share or publicly release your solutions. In group work, be honest about contributions of team members, do not cover for others. If you are using ChatGPT or any other AI system to help, please acknowledge where / how you used it. If you feel overwhelmed or stressed, please come and talk to us!