=========================================================================== CSC 236 Lecture Summary for Week 8 Winter 2008 =========================================================================== Next week: - Term Test 2: Thu 13 Mar, 10-11am, BA *2195* Algorithm complexity, i.e., recurrence relations: deriving them from algorithms, solving them with Master Theorem or repeated substitution, proving bounds on them inductively; algorithm correctness: pre/post conditions, proving correctness of recursive algorithms by induction on the input size, loop invariants, proving partial correctness and termination of iterative algorithms. This does NOT include material on using loop invariants to write code. - Lectures TR1 in SS2108, as usual, except with Robert Danek. - I will be away at a conference starting Tuesday. Special office hours: Mon 10-12 at my office; Tue *3-4* and Wed 2-3 in *BA4290* with TAs -- in addition to regular TA office hour Tue 12-1 in BA2200. Proving loop invariant for example: def pow(x, y): # function precondition: x (- R, y (- N z = 1 m = 0 # loop precondition: x (- R, y (- N, z = 1, m = 0 # LI: m <= y /\ z = x^m while m < y: z *= x m += 1 # loop postcondition: m = y, z = x^y return z # function postcondition: returns x^y How to prove loop invariant? By induction on number of iterations. Base Case: Before loop starts, m_0 = 0 <= y (- N and z_0 = 1 = x^0 = x^{m_0}, so LI_0 is true.. Ind. Hyp.: Let k >= 0 and suppose LI_k holds, i.e., m_k <= y and z_k = x^{m_k}. Ind. Step: To prove LI_{k+1}, consider two cases. Case 1: There is no iteration number k+1. Then, LI_{k+1} equivalent to LI_k (since m_{k+1} = m_k and z_{k+1} = z_k), so LI_{k+1} holds by IH. Case 2: There is an iteration number k+1. Then, m_k < y (loop condition was true) and from algorithm, z_{k+1} = z_k * x and m_{k+1} = m_k + 1. Hence, m_{k+1} = m_k + 1 <= y, and z_{k+1} = z_k * x = x^{m_k} * x = x^{m_k+1} = x^{m_{k+1}} (since z_k = x^{m_k} by IH). Hence, LI_{k+1} holds in all cases. Therefore, by induction, LI holds after each iteration. When loop terminates, m >= y (from the loop test) and m <= y and z = x^m (from LI) so m = y and z = x^m = x^y. This proves loop postcondition, and function is correct. Termination: - Standard method: find expression E involving program variables such that: . E_k (- N for all k; . for all k, E_{k+1} < E_k if loop iterates at least k+1 times (E_{k+1} = E_k otherwise). - By well-ordering, sequence E_0 > E_1 > ... must be finite, i.e., the loop terminates. - Example: What quantity gets smaller? m gets larger, closer to y, so let E = y-m. Then by LI, E_k = y - m_k >= 0 (because m_k <= y), i.e., E_k (- N for all k. Also, if iteration k+1 is performed, E_{k+1} = y - m_{k+1} = y - (m_k + 1) = y - m_k - 1 = E_k - 1 < E_k. Hence, E satisfies both conditions, and loop terminates. General technique for multiple loops: - Consecutive loops (not nested): simply prove separate loop invariants one after the other, where proof of second one can rely on first one being true at the end of the first loop (like having partial pre- and post-conditions in between the loops). - Nested loops: work inside-out. Prove termination and loop invariant for inside loop first, for arbitrary value of the outside loop variables, and use that to prove termination and invariant for outside loop. - Remember that loop for x in [m..n]: ... is equivalent to x = m while x <= n: ... x += 1 This affects how we write invariants and prove termination. More examples: - Section 2.4 on pages 54-58 of textbook. - Extra example posted on course website (on Lectures page). -------------------------- Binary search, iteratively -------------------------- # Pre: A sorted and x comparable with A[0..n-1] (n = len(A)) # Post: 0 <= p <= n and A[0..p-1] < x <= A[p..n-1] What we know at start: A: |_____________________?_____________________| 0 n-1 What we want at end: A: |_______ < x ___________|______ >= x _______| 0 p n-1 In between? Maintain range to search [b..e]. Since A[b..e] are elements we don't know about yet, we must have A[0..b-1] < x <= A[e+1..n]. This will be our loop invariant! A: |___ < x ___|_______?_______|____ >= x _____| 0 b e n-1 Now, we can start writing loop. Initialization? Make "in between" picture same as start: 1. b = 0 2. e = n - 1 Loop condition? Continue as long as range [b..e] not empty, i.e., b <= e. When b > e, "in between" picture looks like end, which is what we want. 3. while b <= e: This is binary search, so compare middle element with x. Let's draw picture to get this right: A: |_ < x _|__________|= x __| 0 b m e n-1 A: |_ < x _|___________|>=x|_________|_ >= x __| 0 b m e n-1 4. m = (b + e) / 2 # integer division 5. if A[m] < x: b = m + 1 6. else: e = m - 1 Does this work? Can prove for all b <= e, b <= m <= e. This guarantees values of b, e at end of iteration satisfy b <= e+1. We add this to loop invariant. Value of p at end? According to picture, p = b = e+1. All together: # Pre: A sorted and x comparable with A[0..n-1] (n = len(A)) 1. b = 0 2. e = n - 1 # LI: 0 <= b <= e+1 <= n and A[0..b-1] < x <= A[e+1..n] 3. while b <= e: 4. m = (b + e) / 2 # integer division 5. if A[m] < x: b = m + 1 6. else: e = m - 1 7. p = b # Post: 0 <= p <= n and A[0..p-1] < x <= A[p..n-1] Wait! What about termination? Hopefully, e and b get closer to each other at each iteration. Actually, E = e+1-b works: - From LI, b <= e+1 guarantees E (- N. - For any iteration, since b <= m <= e, e+1-(m+1) < e+1-m <= e+1-b and (m-1)+1-b < m+1-b <= e+1-b so either way, E becomes smaller.