=========================================================================== CSC 236 Lecture Summary for Week 3 Winter 2008 =========================================================================== ANNOUNCEMENTS: - Assignment 1 is complete (last question was added on Friday). - Next week's office hours changed to T10-12, W2-4, because of Undergraduate Project Showcase on Wed 2-4, BA1230 -- come and find out about undergraduate research projects from last year. - You can submit homework electronically through CDF's 'submit' command -- check the Homework page for details. - Faculty rule: 1-month deadline for remarking requests (after work returned). Everyone OK with chocolate bar proof from last week? Format: Proofs using complete induction sometimes written in style closer to simple induction, using one (or more) base case(s). Only difference then is in form of inductive hypothesis. For example, previous proof's format could look like: 0. Define P(n). 1. Base Case(s): Prove P(1). [Case 1 in proof from last week.] 2. Ind. Hyp.: Let n > 1 (careful: n is next value to prove, not last value proven in base case) and suppose P(1), ..., P(n-1) hold. 3. Ind. Step: Prove P(n). [Case 2 in proof from last week.] 4. Conclusion: By complete induction, \-/ n >= 1, P(n). ------------- Well ordering ------------- Induction "works" because of structure of natural numbers: N is smallest set that contains 0 and successor of any element in N -- smallest because other sets also have this property (e.g., Z, Q, R). Another way to express this property is called Principle of Well Ordering: Every non-empty subset of N contains a smallest element. This applies to *all* non-empty subsets, even infinite ones. Note that this is a property of N -- it does not apply to other sets such as Z, Q, R. Connection with induction? Turns out well ordering, simple induction, and complete induction all equivalent to each other! In other words, it's possible to prove any two of them by assuming the third one. We prove this with "chain of implications" technique: show well ordering -> simple induction, simple induction -> complete induction, and complete induction -> well ordering, i.e., any one principle allows us to conclude the other two. Well ordering -> Simple induction - Suppose well ordering holds, i.e., every non-empty subset of N contains a smallest element. - To prove principle of simple induction, for arbitrary predicate P, suppose P(0) /\ \-/ n, P(n) -> P(n+1). - We prove \-/ n, P(n), by contradiction: Suppose !\-/ n, P(n), i.e., -] n, !P(n). Then, { n (- N : !P(n) } is non-empty subset of N, so by well ordering, it contains smallest element k. - What can we say about k? Since P(0), k > 0. Since k is smallest element such that !P(k) and k > 0, we know P(k-1). However, \-/ n, P(n) -> P(n+1); in particular, P(k-1) -> P(k). Because P(k-1), we have P(k). But this contradicts !P(k)! - By contradiction, \-/ n, P(n). Hence, simple induction holds. Simple induction -> Complete induction - [Only main idea was covered in lecture.] - Suppose simple induction holds, i.e., for arbitrary predicates P, ( P(0) /\ \-/ n, P(n) -> P(n+1) ) -> \-/ n, P(n). - To prove complete induction, for arbitrary predicate P, suppose \-/ n, (\-/ k < n, P(k)) -> P(n) (this is IH (1)). - To prove \-/ n, P(n), let P'(n) = \-/ k <= n, P(k). We prove \-/ n, P'(n) using simple induction. 1. Base Case: Prove P'(0), i.e., \-/ k <= 0, P(k), equivalent to P(0). By IH (1) for n = 0, (\-/ k < 0, P(k)) -> P(0); since \-/ k < 0, P(k) is vacuously true, this means P(0) holds. 2. Ind. Hyp. (2): Let n (- N and suppose P'(n): \-/ k <= n, P(k). 3. Ind. Step: Prove P'(n+1), i.e., \-/ k <= n+1, P(k). By IH (2), \-/ k <= n, P(k) <=> \-/ k < n+1, P(k). By IH (1), (\-/ k < n+1, P(k)) -> P(n+1), so P(n+1). P(n+1) /\ \-/ k < n+1, P(k) -> \-/ k <= n+1, P(k), i.e., P'(n+1). 4. Conclusion: By induction, \-/ n, \-/ k <= n, P(k). - Let n (- N. Then \-/ k <= n, P(k); in particular, P(n). Hence, \-/ n, P(n). Therefore, complete induction holds. Complete induction -> Well ordering - [Only main idea was covered in lecture.] - Suppose complete induction holds, i.e., for arbitrary predicates P, ( \-/ n, (\-/ k < n, P(k)) -> P(n) ) -> \-/ n, P(n). - Let S be an arbitrary subset of N. We show that if S is non-empty, then S contains a smallest element by indirect proof. - Suppose that S does not contain a smallest element, i.e., \-/ n (- N, n (- S -> -] k < n, k (- S. Note that the contrapositive of this statement is \-/ n, (\-/ k < n, k !(- S) -> n !(- S. - Let P(n) = n !(- S. Then, \-/ n, (\-/ k < n, P(k)) -> P(n), by reasoning above, so by complete induction, \-/ n, P(n), i.e., \-/ n, n !(- S, meaning S is empty. - Hence, well ordering holds. ----------------- Induction "traps" ----------------- Each proof below is incorrect in a subtle way. - Define P(n): \sum_{0 <= i <= n} 2^i = 2^{n+1} + 5. We "prove" \-/ n (- N, P(n): . Let n (- N and suppose P(n). . Want to prove P(n+1): \sum_{0 <= i <= n} 2^i = 2^{n+1} + 5. However, \sum_{0 <= i <= n+1} 2^i = \sum_{0 <= i <= n} 2^i + 2^{n+1} = 2^{n+1} + 5 + 2^{n+1} = 2^{n+2} + 5. . Hence, \-/ n (- N, \sum_{0 <= i <= n} 2^i = 2^{n+1} + 5. Problem? Missing base case! Without base case, all that induction proves is \-/ n, P(n) -> P(n+1), but this cannot be used to conclude that P holds for any one specific value of n. Base case may be easy, but it is essential to structure of induction. Moral: Dont forget base case! - Define P(n): In every set of n horses, all horses have the same colour. (Say that every horse has a single "main" colour we are considering.) We "prove" \-/ n (- N, P(n): Base Case: P(0) is vacuously true: in every set of 0 horse, all horses have the same colour. P(1) is true: in every set of 1 horse, all horses have the same colour. Ind. Hyp.: Let n (- N and suppose P(n): in every set of n horses, all horses have the same colour. Ind. Step: Prove P(n+1): in every set of n+1 horses, all horses have the same colour. Let S be a set of n+1 horses, say S = { h_1, h_2, ... h_n, h_{n+1} }. Consider { h_1, h_2, ..., h_n }: this is a set of n horses, so by IH, all horses in that set have the same colour, say A. Now consider { h_2, ..., h_n, h_{n+1} }: this is also a set of n horses, so by IH, all horses in that set have the same colour, say B. Since horses h_2, ..., h_n belong to both sets and cannot have two different main colours, it must be that A = B. This means every horse in S has the same colour. Problem? Base Case is OK, Ind. Hyp. is OK, but Ind. Step makes implicit assumption about n: reasoning only works if n > 1 (if n = 1, reasoning fails for S = { h_1, h_2 } because there is no horse in range h_2 .. h_n). This could be fixed by changing IH to "let n >= 2", but then proof shows P(1) /\ \-/ n >= 2, P(n) -> P(n+1). In this case, IH does not "connect" with base case(s). If we try to "fix" this by adding base case P(2), proof fails because P(2) cannot be proven. Moral: Look out for implicit assumptions in inductive step. - Prove 1^2 + 2^2 + ... + n^2 (- O(n^2). Base Case: 1^2 = 1^2 <= c 2^1 for any c >= 1. Ind. Hyp.: Let n >= 1 and suppose 1^2 + ... + n^2 (- O(n^2), i.e., 1^2 + ... + n^2 <= c n^2 for some constant c > 0. Ind. Step: Prove 1^2 + ... + (n+1)^2 (- O((n+1)^2). However, 1^2 + ... + (n+1)^2 = 1^2 + ... + n^2 + (n+1)^2 <= c n^2 + (n+1)^2 <= c (n+1)^2 + (n+1)^2 <= (c+1) (n+1)^2 so 1^2 + ... + (n+1)^2 <= (c+1) (n+1)^2, for constant c+1. Problem? Predicate P not carefully defined, so not used properly. Recall that "1^2 + ... + n^2 (- O(n^2)" stands for -] c > 0, -] b, \-/ n >= b, 1^2 + ... + n^2 <= c n^2. But this is NOT a statement about a single value of n (it contains \-/ n), so not a valid P(n)! This shows up in "proof" above because value of constant *changes* between IH and ind. step -- but then it's not constant! To prove 1^2 + ... + n^2 (- O(n^2) by induction would mean: Pick values of c > 0 and b. Prove \-/ n >= b, 1^2 + ... + n^2 <= c n^2 using induction (where values of c, b are fixed). However, this is not possible (SUM i^2 !(- O(n^2)). Moral: Define P(n) carefully!