=========================================================================== CSC 236 Lecture Summary for Week 3 Fall 2007 =========================================================================== ------------- Well ordering ------------- Induction "works" because of structure of natural numbers: N is smallest set that contains 0 and successor of any element in N -- smallest because other sets also have this property (e.g., Z, Q, R). Another way to express this property is called Principle of Well Ordering: Every non-empty subset of N contains a smallest element. This applies to *all* non-empty subsets, even infinite ones. Note that this is a property of N -- it does not apply to other sets such as Z, Q, R. Example: Proof of division algorithm: \-/ m, \-/ n > 0, -] r, -] q, r < n /\ m = q n + r. Let m in N, and n > 0 in N. Let R = { r in N : -] q, m = q n + r }. Then, R is not empty because m in R (m = 0 n + m), so R contains a smallest element r by well ordering. Claim: r < n -- otherwise, r >= n and m = q n + r would imply m = q n + n + r - n = (q+1) n + (r-n), so r-n in R and r-n < r (because n > 0), contradicting r is smallest in R. Hence, -] r, -] q, r < n /\ m = q n + r, as desired. Connection with induction? Turns out well ordering, simple induction, and complete induction all equivalent to each other! In other words, it's possible to prove any two of them by assuming the third one. We prove this with "chain of implications" technique: show well ordering -> simple induction, simple induction -> complete induction, and complete induction -> well ordering. Any one principle allows us to conclude the other two. Well ordering -> Simple induction - Suppose well ordering holds, i.e., every non-empty subset of N contains a smallest element. - To prove principle of simple induction, for arbitrary predicate P, suppose P(0) /\ \-/ n, P(n) -> P(n+1). - We prove \-/ n, P(n), by contradiction: Suppose ~ \-/ n, P(n), i.e., -] n, ~ P(n). Then, { n in N : ~ P(n) } is non-empty subset of N, so by well ordering, it contains smallest element k. - What can we say about k? Since P(0), k > 0. Since k is smallest element such that ~ P(k) and k > 0, we know P(k-1). However, \-/ n, P(n) -> P(n+1); in particular, P(k-1) -> P(k). Because P(k-1), we have P(k). But this contradicts ~ P(k)! - By contradiction, \-/ n, P(n). Hence, simple induction holds. Simple induction -> Complete induction - [Only main idea was covered in lecture.] - Suppose simple induction holds, i.e., for arbitrary predicates P, ( P(0) /\ \-/ n, P(n) -> P(n+1) ) -> \-/ n, P(n). - To prove complete induction, for arbitrary predicate P, suppose \-/ n, (\-/ k < n, P(k)) -> P(n) (this is IH (1)). - To prove \-/ n, P(n), let P'(n) = \-/ k <= n, P(k). We prove \-/ n, P'(n) using simple induction. 1. Base Case: Prove P'(0), i.e., \-/ k <= 0, P(k), equivalent to P(0). By IH (1) for n = 0, (\-/ k < 0, P(k)) -> P(0); since \-/ k < 0, P(k) is vacuously true, this means P(0) holds. 2. Ind. Hyp. (2): Let n in N and suppose P'(n): \-/ k <= n, P(k). 3. Ind. Step: Prove P'(n+1), i.e., \-/ k <= n+1, P(k). By IH (2), \-/ k <= n, P(k) <=> \-/ k < n+1, P(k). By IH (1), (\-/ k < n+1, P(k)) -> P(n+1), so P(n+1). P(n+1) /\ \-/ k < n+1, P(k) -> \-/ k <= n+1, P(k), i.e., P'(n+1). 4. Conclusion: By induction, \-/ n, \-/ k <= n, P(k). - Let n in N. Then \-/ k <= n, P(k); in particular, P(n). Hence, \-/ n, P(n). Therefore, complete induction holds. Complete induction -> Well ordering - [Only main idea was covered in lecture.] - Suppose complete induction holds, i.e., for arbitrary predicates P, ( \-/ n, (\-/ k < n, P(k)) -> P(n) ) -> \-/ n, P(n). - Let S be an arbitrary subset of N. We show that if S is non-empty, then S contains a smallest element by indirect proof. - Suppose that S does not contain a smallest element, i.e., \-/ n in N, n in S -> -] k < n, k in S. Note that the contrapositive of this statement is \-/ n, (\-/ k < n, k notin S) -> n notin S. - Let P(n) = n notin S. Then, \-/ n, (\-/ k < n, P(k)) -> P(n), by reasoning above, so by complete induction, \-/ n, P(n), i.e., \-/ n, n notin S, so S is empty. - Hence, well ordering holds. ----------------- Induction "traps" ----------------- Each proof below is incorrect in a subtle way. - Define P(n): SUM_{0 <= i <= n} 2^i = 2^{n+1} + 5. We "prove" \-/ n in N, P(n): . Let n in N and suppose P(n). . Want to prove P(n+1): SUM_{0 <= i <= n} 2^i = 2^{n+1} + 5. However, SUM_{0 <= i <= n+1} 2^i = SUM_{0 <= i <= n} 2^i + 2^{n+1} = 2^{n+1} + 5 + 2^{n+1} = 2^{n+2} + 5. . Hence, \-/ n in N, SUM_{0 <= i <= n} 2^i = 2^{n+1} + 5. Problem? Missing base case! Without base case, all that induction proves is \-/ n, P(n) -> P(n+1), but this cannot be used to conclude that P holds for any one specific value of n. Base case may be easy, but it is essential to structure of induction. - Define P(n): In every set of n horses, all horses have the same colour. (Suppose that every horse has a "main" colour we are considering.) We "prove" \-/ n in N, P(n): Base Case: P(0) is vacuously true: in every set of 0 horse, all horses have the same colour. P(1) is true: in every set of 1 horse, all horses have the same colour. Ind. Hyp.: Let n in N and suppose P(n): in every set of n horses, all horses have the same colour. Ind. Step: Prove P(n+1): in every set of n+1 horses, all horses have the same colour. Let S be a set of n+1 horses, say S = { h_1, h_2, ... h_n, h_{n+1} }. Consider { h_1, h_2, ..., h_n }: this is a set of n horses, so by IH, all horses in that set have the same colour, say A. Now consider { h_2, ..., h_n, h_{n+1} }: this is also a set of n horses, so by IH, all horses in that set have the same colour, say B. Since horses h_2, ..., h_n belong to both sets and cannot have two different main colours, it must be that A = B. This means every horse in S has the same colour. Problem? Base Case is OK, Ind. Hyp. is OK, but Ind. Step makes implicit assumption about n: reasoning only works if n > 1 (if n = 1, reasoning fails for S = { h_1, h_2 } because there is no horse in range h_2 .. h_n). This could be fixed by changing IH to "let n >= 2", but then proof shows P(1) /\ \-/ n >= 2, P(n) -> P(n+1). In this case, IH does not "connect" with base case(s). If we try to "fix" this by adding base case P(2), proof fails because P(2) cannot be proven. - Prove 1^2 + 2^2 + ... + n^2 in O(n^2). Base Case: 1^2 = 1^2 <= c 2^1 for any c >= 1. Ind. Hyp.: Let n >= 1 and suppose 1^2 + ... + n^2 in O(n^2), i.e., 1^2 + ... + n^2 <= c n^2 for some constant c > 0. Ind. Step: Prove 1^2 + ... + (n+1)^2 in O((n+1)^2). However, 1^2 + ... + (n+1)^2 = 1^2 + ... + n^2 + (n+1)^2 <= c n^2 + (n+1)^2 <= c (n+1)^2 + (n+1)^2 <= (c+1) (n+1)^2 so 1^2 + ... + (n+1)^2 <= (c+1) (n+1)^2, for constant c+1. Problem? Predicate P not carefully defined, so not used properly. Recall that "1^2 + ... + n^2 in O(n^2)" stands for -] c > 0, -] b, \-/ n >= b, 1^2 + ... + n^2 <= c n^2. But this is NOT a statement about a single value of n (it contains \-/ n), so not a valid P(n)! This shows up in "proof" above because value of constant *changes* between IH and ind. step -- but then it's not constant! To prove 1^2 + ... + n^2 in O(n^2) by induction would mean: Pick values of c > 0 and b. Prove \-/ n >= b, 1^2 + ... + n^2 <= c n^2 using induction (where values of c, b are fixed). However, this is not possible (SUM i^2 not in O(n^2)). --------------------------------------------------------------------------- Additional example (not covered in lecture). Use well ordering to prove that SUM_{0 <= i <= n} 2^i = 2^{n+1}-1. For a contradiction, suppose that there are values of n in N for which SUM_{0 <= i <= n} 2^i =/= 2^{n+1}-1. Then, by the principle of well ordering, there is a smallest such value, say k. Note that k =/= 0 because SUM_{0 <= i <= 0} 2^i = 2^0 = 1 = 2^{0+1}-1. Since k > 0, k-1 in N so SUM_{0 <= i <= k-1} 2^i = 2^k-1 (because k is the smallest value for which equality does not hold). Then, SUM_{0 <= i <= k} 2^i = SUM_{0 <= i <= k-1} 2^i + 2^k = 2^k-1 + 2^k = 2*2^k - 1 = 2^{k+1} - 1, which contradicts SUM_{0 <= i <= k} 2^i =/= 2^{k+1}-1. Hence, there is no value of n in N for which SUM_{0 <= i <= n} 2^i =/= 2^{n+1}-1, i.e., SUM_{0 <= i <= n} 2^i = 2^{n+1}-1 for all n in N. NOTES: - Proof contains "base case" and "inductive step" from induction proof -- not surprising because we are proving the same statement, just written up differently. - Generic proof of \-/ n, P(n) using well ordering looks like this: For a contradiction, suppose -] n, ~ P(n). Well ordering implies there is a smallest such value k (i.e., ~ P(k) but P(i) for any i < k). However, k > 0 because we can prove P(0) directly [insert proof here]. Since k > 0, k-1 >= 0 and P(k-1) (because k smallest such that ~ P(k)). However, we can show P(k-1) -> P(k) [insert proof here], which together with P(k-1) contradicts ~ P(k). Hence, \-/ n, P(n).