=========================================================================== CSC 236 Lecture Summary for Week 1 Winter 2008 =========================================================================== Course information sheet. What do you expect from this course and from me? >> Let me know! What do I expect from this course and from you? - Material is not that difficult, but very abstract: you must keep up in order to understand. In particular, do the readings and take the time to work on the exercises/assignments carefully. Also, please let me know when things don't make sense so that I can provide additional explanations or examples. - Main reason people do poorly: not enough time spent on this course. Even though there is no programming, you might want to spend as much time on this course as on programming courses, reading notes and working on examples to understand the material. It takes time to sink in and make sense: in general, waiting to the last minute won't work! - Please don't hesitate to get in touch with me with any concerns or suggestions you may have about any aspect of the course: if there are aspects you particularly like, I want to know so that I can keep them the same; if there are aspects you particularly dislike, I want to know so that I can make changes (or explain my reasons for having things this way) -- don't wait until the official course evaluations at the end of term, when it's too late to make a difference now. - LAPTOPS: Because of a bad experience in the past, I'd like to ask you to please not show up just for the sake of showing up and spend your time working/chatting/playing (rather than paying attention and taking notes) -- it's a distraction for other students and kind of insulting for me. I don't take attendance, and my lecture notes are posted online: I would think it more considerate for someone to miss a lecture than to show up with no intention of participating. ------------ Introduction ------------ Induction = Recursion Topics: - Induction (simple, complete, well-ordering, structural). Chapters 1, 4 [3 weeks] - Algorithm complexity and recurrence relations. Chapter 3 [2 weeks] - Algorithm correctness. Chapter 2 [2 weeks] - Regular languages, finite-state automata, and regular expressions. Chapter 7 [3 weeks] - Context-free languages, context-free grammars, and pushwon automata. Chapter 8 [3 weeks] Consider: a = 1/5 while true: print a a = (1 + a) / 2 - What does this print? - Trace: 1/5, 3/5, 4/5, 9/10, 19/20, ... (assume perfect arithmetic) - Conjectures? 1. every value < 1 2. values increasing 3. limit = 1 (outside scope of this course!) - Proofs? First, try to convince ourselves. 1. (a) 1/5 < 1 (b) if a < 1, then (1 + a) / 2 is average of 1 and a so < 1 (arithmetic: (1 + a) / 2 < (1 + 1) /2) (c) so it must be the case that a < 1 always -> This is a proof by induction! - Conjecture 2: Is it always the case that a < (1 + a) / 2? Yes: a < 1 -> (a + a) / 2 < (1 + a) / 2 More formally. - In math, variable = unknown but fixed quantity. In CS, variable = name of a memory location, whose content can change. - How to express mathematically (i.e., precisely) changing values of an algorithm's variable? - Convention: Let a_n represent value of a at the end of n complete iterations of the loop (i.e., just before loop test is evaluated for (n+1)st time); in particular, a_0 = value before loop starts. - Algorithm's computation simply evaluates the sequence: a_0 = 1/5 a_{n+1} = (1 + a_n) / 2 for all n (- N [ASCII notation: (- = element of] - Conjectures can be written: [ASCII notation: \-/ = for all] 1. \-/ n (- N, a_n < 1 2. \-/ n (- N, a_n < a_{n+1} - Notice: . sequence defined inductively/recursively; . conjecture 1 is about individual elements, and was proved using induction; . conjecture 2 is about consecutive elements, and was proved directly (making use of conjecture 1). - Symbolic proofs: Proof of Conjecture 1: . a_0 = 1/5 < 1. . Let n (- N and suppose a_n < 1. Then, a_{n+1} = (1 + a_n) / 2 < (1 + 1) / 2 = 1. Hence, \-/ n (- N, a_n < 1 -> a_{n+1} < 1. . So, because a_0 < 1 and \-/ n (- N, a_n < 1 -> a_{n+1} < 1, we conclude \-/ n (- N, a_n < 1 Principle of Simple Induction: For a predicate P(n), ( P(0) /\ \-/ n (- N, P(n) -> P(n+1) ) -> \-/ n (- N, P(n). Claim: This principle is "self-evident". We will never attempt to "prove" it; rather we will only make use of it. Making use of an implication A -> B? Prove A, then conclude B. In other words, in order to prove \-/ n (- N, P(n) using induction, we prove instead P(0) /\ \-/ n (- N, P(n) -> P(n+1). When making use of induction, we never directly prove \-/ n (- N, P(n). ---------------------------------------------- Standard format for proofs by simple induction ---------------------------------------------- 0. Define precisely P(n), the statement for which we want to prove \-/ n (- N, P(n). CAREFUL: We are not proving P(n), but \-/ n (- N, P(n), so the statement P(n) should NOT contain any quantifier on variable n, i.e., P(n) must be a statement about a single value n, which is not the case if P(n) contains "\-/ n" or "-] n". [ASCII notation: -] = there exists] 1. Base Case: Prove P(0). 2. Inductive Hypothesis (IH): Let n (- N, and suppose P(n). 3. Inductive Step: Prove P(n+1). 4. Conclusion: By induction, \-/ n (- N, P(n). Example 1: Prove that 12^n - 1 is divisible by 11 for all n (- N. 0. P(n) = -] k (- N, 12^n - 1 = 11 k 1. Base Case: P(0) = -] k (- N, 12^0 - 1 = 11 k. But 12^0 - 1 = 1 - 1 = 0 = 11 * 0, so P(0) holds by picking k = 0. Formally, in the style of CSC 165: Let k = 0. Then k (- N and 12^0 - 1 = 0 = 11 k. Hence, -] k (- N, 12^0 - 1 = 11 k. (In this course, we use the first "shorthand" style rather than the detailed style from CSC 165.) 2. IH: Let n (- N and suppose P(N), i.e., -] k (- N, 12^n - 1 = 11 k. 3. Inductive Step: Need to prove -] k' (- N, 12^{n+1} - 1 = 11 k'. However, 12^{n+1} - 1 = 12 * 12^n - 1 = 11 * 12^n + 12^n - 1 = 11 * 12^n + 11 k (where k is the value we know exists by the IH) = 11 * (12^n + k) so P(n+1) holds by picking k' = 12^n + k. 4. Hence, by induction, \-/ n (- N, P(n), i.e., 12^n - 1 is a multiple of 11 for all n (- N. Example 2: Given an unlimited supply of 4c and 7c stamps, what amounts of postage can be made exactly? - Explore: Try amounts until pattern emerges. Impossible: 1c, 2c, 3c, 5c, 6c, 9c, 10c, 13c, 17c Possible: 4c, 7c, 8c, 11c, 12c, 14c, 15c, 16c, 18c, 19c, 20c, 21c, ... - Conjecture: Every amount of 18c or more can be made exactly. - Proof? By induction. [I did not have time to cover the rest of the proof in detail during lecture, so I provide it here in a "question and answer" format. See if you can answer the questions on your own, then use the answers to verify your understanding -- which you will need in order to complete the homework.] Q: What's the first step? 0. Define P(n) precisely. Q: What does does it mean to "make an amount of postage of n cents using only 4c and 7c stamps"? How do we express it precisely? A: -] a (- N, -] b (- N, 4a + 7b = n Q: Now, what should P(n) be? A1: P(n): n >= 18 -> -] a (- N, -] b (- N, 4a + 7b = n. A2: P(n): -] a (- N, -] b (- N, 4a + 7b = n + 18. A3: P(n): -] a (- N, -] b (- N, 4a + 7b = n, except we prove \-/ n >= 18, P(n) instead of \-/ n (- N, P(n). Each of these will require slightly different proof formats, but the content will be the same (and the differences slight, as we'll see below). We'll pick the first one to do, because it allows us to review important aspects of working with implications. Q: What's the next step? 1. Base Case: Prove P(0). Q: What's P(0) again? A: 0 >= 18 -> -] a (- N, -] b (- N, 4a + 7b = 0. Q: How do we prove this? A: This is vacuously true (i.e., 0 >= 18 is false). Recall that when A is false, A -> B is true, and this is called "vacuously true" because it tells us nothing about the truth of B. Q: What's the next step? 2. Inductive Hypothesis (IH): Q: What should this be? A: Let n (- N and suppose P(n), i.e., n >= 18 -> -] a (- N, -] b (- N, 4a + 7b = n. Q: What's the next step? 3. Inductive Step: Prove P(n+1), i.e., n+1 >= 18 -> -] a' (- N, -] b' (- N, 4a' + 4b' = n+1 (using a', b' to avoid possible confusion with the variables in the IH). Q: How do we prove this? A: It's an implication, so start with: Suppose n+1 >= 18. Q: Can we use the IH directly? A: No: n+1 >= 18 means n >= 17, which doesn't match. Q: How can we work around this? A: Break up the inductive step into cases: n+1 = 18 (which we'll have to prove directly) and n+1 > 18 (where we'll be able to use the IH). Consider the following cases: either n+1 = 18 or n+1 > 18. Q: If n+1 = 18, how do we prove P(n+1)? Case 1: If n+1 = 18, then n+1 = 18 = 4 + 14 = 4*1 + 7*2, so P(n+1) holds by picking a' = 1, b' = 2. Q: If n+1 > 18, how do we prove P(n+1)? Case 2: If n+1 > 18, then n >= 18 and the IH allows us to conclude -] a (- N, -] b (- N, 4a + 7b = n. Hence, n+1 = 4a + 7b + 1. Q: What next? A: We need a bit of a trick! Consider the following cases: either b > 0 or b = 0. Q: Now, can you figure out how to prove the case b > 0? Sub-case a: If b > 0, then n+1 = 4a + 7b + 1 = 4a + 7(b-1) + 7 + 1 = 4a + 7(b-1) + 8 = 4(a+2) + 7(b-1) so P(n+1) holds by picking a' = a+2, b' = b-1. Q: Now, can you figure out how to prove the case b = 0? Sub-case b: If b = 0, then n+1 = 4a + 1. Since n >= 18, this means 4a >= 18, which implies a >= 5 (since a (- N). Then, n+1 = 4a + 1 = 4(a-5) + 20 + 1 = 4(a-5) + 7*3 so P(n+1) holds by picking a' = a-5, b' = 3. In every sub-case, P(n+1) holds. In every case, P(n+1) holds. Q: At this point, what have we proved exactly? A: \-/ n (- N, P(n) -> P(n+1) (just completed) and P(0) (earlier). Q: What do we want to prove again? A: \-/ n (- N, P(n). Q: How do we get there? A: *Apply* the principle of simple induction. Q: What's the next step? 4. Conclusion: By induction, \-/ n (- N, P(n), i.e., \-/ n (- N, n >= 18 -> -] a (- N, -] b (- N, 4a + 7b = n, i.e., it is possible to make every amount of postage of 18c or more using only 4c and 7c stamps. Now, look back over the proof. Q: Notice anything about Case 1 and Case 2 in the proof of our inductive step? A: They look an awful lot like the base case and inductive step in some other proof by induction. That's because they are! If we used the third definition of P(n) and used a "modified" version of induction with a base case of n = 18 and an IH that started with "let n (- N and n >= 18", we would get exactly Case 1 as our base case and Case 2 as our inductive step. And we would conclude exactly the same thing in the end. Q: What about the second definition of P(n)? A: It would look almost exactly like the proof using the third definition, except with the "standard" structure for simple induction, and with "n+18" instead of "n" on the left-hand side of the equations.