Robotics: Science and Systems

Mars Exploration Rover


This course provides an introduction to robotic systems from a computational perspective. A robot is regarded as an intelligent computer that can use sensors and act on the world. We will consider the definitional problems in robotics and look at how they are being solved in practice and by the research community. The emphasis is on algorithms, probabilistic reasoning, optimization, inference mechanisms, and behavior strategies, as opposed to electromechanical systems design. This course aims to help students improve their probabilistic modeling skills and instill the idea that a robot that explicitly accounts for its uncertainty works better than a robot that does not.

Course Details

Florian Shkurti, x=csc477-instructor
Lectures: Wed, 3-5pm ET
Office Hours: Thu, 3-4pm ET
Teaching Assistants
Yash Kant
Office Hours: Tue, 4-5pm ET
Tutorial: Wed, 6-7pm ET
Philip (Yizhou) Huang
Office Hours: Fri, 10-11am ET
Tutorial: Wed, 5-6pm ET, y=csc477-tas
Course Links
Zoom links have been posted on Quercus
Announcements will be posted on Quercus
Discussions will take place on Piazza
Anonymous feedback form for suggested improvements
Course Syllabus

Course Description

This course will broadly cover the following areas:

  • Kinematics and Dynamics: how can we model robotic systems using approximate physical models that enable us to make predictions about how robots move in response to given commands?
  • Feedback Control and Planning: how can we compute the commands that can bring a robotic system from its current state to a desired state?
  • Mapping: how can we combine noisy measurements from sensors with the robot’s pose to build a map of the environment?
  • State Estimation: the state of the robot is not always directly measurable/observable. How can we determine the relative weighs of multiple sensor measurements in order to form an accurate estimate of the (hidden) state?
  • Geometry of Computer Vision: how can modeling pixel projections on an RGB camera help us infer the 3D structure of the world? How can we triangulate points seen from two cameras? How can we estimate the camera’s pose (and therefore the robot’s) while it is moving in the environment?


Week Date Lecture Tutorial/Practical Slides Reading
1 Sep 8 Introduction
Motivation, logistics, rough description of assignments, sense-plan-act paradigm.

Sensors and Actuators
Camera, LiDAR, tactile, IMU, depth, GPS, Hall-effect sensors, encoders, RGBD. Pulse-Width Modulation. Motors.


Dudek & Jenkin 3.1.1,4, 3.2-3, 4.1-8, 4.10, 5.1.1
Optional: Mike Langer's notes
2 Sep 15 Kinematics
Frames of reference. Rotation representations. Homogeneous coordinates and transformations. Rigid body motion.

Dynamical systems and control. Examples: Dubins car, differential drive car, unicycle, pendulum, cartpole, quadcopter. Holonomic vs. non-holonomic systems.

Intro to ROS pdf, pptx

Paul Furgale: robot pose
Lavalle 13.1
Dudek & Jenkin 3.1.5,6
3 Sep 22 PID Control
Tuning, cascading PID, advantages and drawbacks.

Artificial Potential Fields and Obstacle Avoidance
Implementation issues, navigation functions. Vector-field histogram (VFH), dynamic window approach (DWA).

Linear algebra refresher pdf, pptx

Optional: Astrom and Hagglund, Ch. 2

Lavalle Ch. 8.4
Dudek & Jenkin 6.3.4
Optional: Howie Choset's notes
4 Sep 29 Linear Quadratic Regulator (LQR)
Computing optimal actions for linear dynamical systems with quadratic cost-to-go functions.

Probability refresher pdf, pptx, code Optional: Stephen Boyd's LQR notes and examples
5 Oct 6 Planning
Dijkstra, A*, Rapidly-exploring Random Trees (RRT), Probabilistic RoadMaps (PRM)
Intro to numpy, Implementing LQR pdf, pptx Blog post on A*
Udacity Lesson 4
Lavalle 5.5, 5.6
Oct 13 Reading week

6 Oct 20 Map Representations and Map Alignment
Occupancy grids, Octrees, Voronoi Graph, Homotopy Classes. Map alignment with known or unknown correspondences. Iterative Closest Point (ICP).

Occupancy Grid Mapping With Known Robot Poses
Log-odds ratio, Probabilistic dynamics and measurement models, Bayesian estimation.

Derivations for occupancy grid mapping pdf, pptx Pieter Abbeel's notes Pieter Abbeel's notes
Probabilistic Robotics Ch. 2 and Ch. 9
7 Oct 27 Maximum Likelihood, Least Squares Estimation, Maximum A Posteriori Estimation
Least squares as a special case of maximum likelihood estimation on Gaussian models.

Expectation and Covariance. Geometric interpretation of the covariance matrix. Nonlinear Least Squares formulation of the Simultaneous Localization And Mapping (SLAM) problem.

Demonstrating gMapping pdf, pptx Udacity Lesson 6
Probabilistic Robotics Ch. 11
8 Nov 3 Kalman Filter
Bayes' rule on Gaussian distributions. Example of 1D Kalman Filter.

Bayes' Filter and Kalman Filter
Kalman Filter as a special case of Bayes' Filter. Examples of 2D and 4D Kalman Filter. General prediction and update equations.

pdf, pptx Udacity Lesson 2
Kalman Filter, Illustrated, Probabilistic Robotics Ch. 2,3
9 Nov 10 Extended Kalman Filter (EKF)
Bayes' Filter and nonlinear transformations. Monte Carlo sampling vs. Linearization. EKF prediction and update equations. Examples: EKF Localization and EKF SLAM.

Basic Kalman Filter implementation pdf, pptx Cyrill Stachniss' intro to EKF
Cyrill Stachniss' intro to EKF-SLAM
Probabilistic Robotics Ch. 2,3
10 Nov 17 Particle Filter
Representing multimodal distributions. Particle propagation and resampling. Pathologies of particle filter. Importance Sampling. Examples: Markov localization in a known map. FastSLAM.

pdf, pptx Udacity Lesson 3
Optional: Thrun's paper on PF
11 Nov 24 Camera Optics and Multi-view Geometry
Pinhole cameras, lenses, perspective projection. Aperture, focal length, exposure time, depth-of-field. Structure from Motion.
pdf, pptx Optional: James Tompkin's notes.
Sanja Fidler's notes on depth from stereo
12 Dec 01 Visual odometry and Visual SLAM
Epipolar constraints. Depth from stereo disparity for parallel cameras. Triangulation as a least-squares problem. Scale issues in visual odometry with a single camera. Visual SLAM vs. structure from motion.
pdf, pptx Optional: James Tompkin's notes on stereo and SfM.
Sanja Fidler's notes on depth from stereo
13 Dec 8 Study break, beginning of exams
Review for final exam


Marking scheme

Recommended, but optional, textbooks

Related courses