CSC 310: Information Theory

Lecture Slides

Lecture 1a: Introduction to Data Compression.

Lecture 1b: Introduction to Error Correction.

Lecture 2: Unique and Instantaneous Decodability. (Updated Jan 13) Supplementary notes

Lecture 3: Proving the Kraft-McMillan Inequalities. Supplementary notes

Lecture 4: Entropy and Huffman Codes. Supplementary notes

Lecture 5: Block Coding and Shannon's First Theorem. Supplementary notes

Lecture 6: Typical Sets and Arithmetic Coding. (Updated Jan 29) Supplementary notes

Lecture 7: Stream Codes and Arithmetic Coding. Supplementary notes

Lecture 8: Arithmetic Coding - Practical Details. (Updated Feb 14) Supplementary notes

Lecture 9: Adaptive Probability Models. (Updated Feb 12) Supplementary notes

Lecture 10: Prediction by Partial Match (PPM).

Lecture 12: Information Channels.

Lecture 13: Real-World Examples.

Lecture 14: Channel Capacity.

Lecture 15: Linear Codes.

Lecture 16: Equivalent Codes and Systematic Forms.

Lecture 17: Proving Shannon's Second Theorem.

Most of the supplementary notes refer to the lecture slides (as indicated at the top left corner of each page of the notes).

The lecture slides are courtesy of Sam Roweis and Radford Neal.