CSC2206 Lecture Slides

Overview
Review Probability Theory
Proofs Weak and Strong Law of Large Numbers
Poisson Processes
Renewal Processes
Wald's Equality
Renewal Reward Processes
Finite State Markov Chains
Perron-Frobenius Theory
Markov Reward Process
Supplement: Relative Gain
Supplement: Review for Midterm
Markov Chains with Countable State Space
Markov Chains with Countable State Space - Steady State Distribution
Branching Processes
Birth-Death Processes
Reversibility
Semi-Markov Processes
Markov Processes
Markov Decision Processes
Summary