CSC 310 - Information Theory

Fall 2009


Outline of Lectures

Lecture 25
Lecture 24
Lecture 23
Lecture 22
Lecture 21
Lecture 20
Lecture 19
Lecture 18
Lecture 17
Lecture 16
Lecture 15
Lecture 14
Lecture 13
Lecture 12
Lecture 11
Lecture 10
Lecture 9
Lecture 8
Lecture 7
Lecture 6
Lecture 5
Lecture 4
Lecture 3
Lecture 2
Lecture 1


Lecture 25

  • Review: Basic Coding Theory (and additional examples)

Lecture 24

  • Review: Basic Information Theory (and additional examples)

Lecture 23

  • More on linear codes
  • Syndrome decoding

Lecture 22

  • Linear Codes: introduction, and "why linear codes?"
  • Distance and linear relations
  • Hamming Codes

Lecture 21

  • Gilbert-Varshamov bound
  • This week's tutorial: review error correcting codes & Hadamard Codes

Lecture 20

  • Introduction to error-correcting codes
  • Hamming's sphere packing bound

Lecture 19

  • More details on the proof of Shannon's Fundamental Theorem
  • Remarks on the hardness of decoding Random Linear Codes & applications
  • The converse of Shannon's Fundamental Theorem (more details in tutorial)

Lecture 18

  • Proof outline of Shannon's Fundamental Theorem
  • This week's tutorial: detailed proof of Shannon's Fundamental Theorem

Lecture 17

  • Unreliable channels
  • Maximum likelihood decoding & nearest neighbor decoding
  • Statement of Shannon's 2nd Theorem (Shannon's Fundamental Theorem)

Lecture 16

  • Examples on reliable communication channels

Lecture 15

  • Mutual information and capacity - examples

Lecture 14

  • System entropies
  • Binary Symmetric Channel (explicit calculations)

Lecture 13

  • Continuing with: Introduction to information channels (Chapter 4)

Lecture 12

  • Introduction to information channels (Chapter 4)

Lecture 11

  • Conclude Chapter 3 (in tutorial: Shannon-Fano coding)

Lecture 10

  • More on Shannon's First Theorem and entropy

Lecture 9

  • "Shannon's First Theorem": Shannon's theorem for noiseless communication

Lecture 8

  • Introduction to entropy (Chapter 3)

Lecture 7

  • Optimal codes (Chapter 2)

Lecture 6

  • Proof of McMillan's Theorem.

Lecture 5

  • Continuing with Kraft's and McMillan's inequalities.
  • Tutorial (on Friday): Sardinas-Paterson Theorem.

Lecture 4

  • Source coding: uniquely decodable and instantaneous codes (Chapter 1)

Lecture 3

  • Continuing with Probability and Linear Algebra prerequisites.

Lecture 2

  • Probability and Linear Algebra prerequisites.

Lecture 1

  • What is information and coding theory? High-level and intuitive presentation of the main concepts and issues.
  • Mention (mostly in bullet-like form) the Probability and Linear Algebra prerequisites.



Last Modified: December 8, 2009