===========================================================================
CSC 263H               Tutorial Summary for Week  1             Winter 2004
===========================================================================

--------------------------
Indicator random variables
--------------------------

Indicator random variables are a powerful and easy to use tool that can
simplify the computation of expected values tremendously.

For example, suppose that we want to compute the expected number of coins
that come up "heads" when we throw n fair coins.  (Intuitively, this should
be n/2, but we want to figure out how to obtain this value rigorously.)

According to the definition of expectation, if we let X be the random
variable denoting the number of coins that come up heads, then X can take
on values ranging from 0 to n so this is equal to:

            n
    E[X] = sum ( i * Pr[X=i] )
           i=0

What is Pr[X=i]?  By simple combinatorics, it is equal to the number of
ways of picking the i coins that come up heads divided by the total number
of outcomes when tossing n coing, i.e., (n choose i) / 2^n.

Hence we get the following expression:

            n
    E[X] = sum ( i * (n choose i) / 2^n )
           i=0
                                         n
         = ( 0 * (n choose 0) / 2^n ) + sum ( i * n!/(i!(n-i)!) / 2^n )
                                        i=1
                  n
         = n/2 * sum ( (n-1)!/((i-1)!(n-i)!) / 2^{n-1} )
                 i=1
                 n-1
         = n/2 * sum ( (n-1 choose j) * (1/2)^{n-1-j} * (1/2)^j )
                 j=0

         = n/2 * (1/2 + 1/2)^{n-1} = n/2.

Evaluating this sum is not trivial, even if we recognize immediately that
we have to use the binomial formula.

Compare this with the following idea: define a set of random variables
X_1, ..., X_n as follows:

          { 1  if coin i comes up heads,
    X_i = {
          { 0  if coin i comes up tails.

These are "indicator" random variables.  Then, we immediately have that
X = X_1 + X_2 + ... + X_n, and by linearity of expectation:

               n           n
    E[X] = E[ sum X_i ] = sum E[X_i]
              i=1         i=1

Finally, E[X_i] is very easy to compute: by definition,

    E[X_i] = 1 * Pr[X_i=1] + 0 * Pr[X_i=0] = Pr[X_i=1]

and in this case, Pr[X_i=1] = Pr[coin i comes up heads] = 1/2, so we
immediately get that

            n            n
    E[X] = sum E[X_i] = sum (1/2) = n/2.
           i=1          i=1

Notice how much simpler the answer was to compute using indicator random
variables than the definition of expectation.  The trick to using indicator
random variables is to define them properly so that they help to simplify
the computation.

------
Review
------

Possible topics to review (depending on what students say they need the
most help with):

  - This week's lecture notes (available on course website).

  - Asymptotic notation (section 1.2 of the textbook).
    (Questions R-1.19 to R-1.23, and C-1.9 in the textbook.)

  - Basic complexity analysis (example in section 1.4 of the textbook
    -- suggest they read section 1.1 if they need more basic stuff).
    (Questions R-1.10 to R-1.14 in the textbook.)

  - Basic probability theory (part of section 1.3 of the textbook).