=========================================================================== CSC 236 Lecture Summary for Week 5 Winter 2008 =========================================================================== Recall recurrence for worst-case running time of RecBinSearch: { 7 if n = 1 T(n) = { { 9 + max{ T(ceil(n/2)), T(floor(n/2)) + 1 } if n > 1 Already proved: T(n) (- \Omega(log n). (b) T(n) (- O(log n): T(n) <= 10 lg(n-1) + 17 for all n >= 2. [How to come up with exact expression? See detailed example below and come back to this one to work it out.] Base Case: - T(2) = 9 + max{T(1), T(1)+1} = 9 + 7+1 = 17 <= 17 = 10 * 0 + 17 = 10 lg(2-1) + 17. [This is where constant 17 comes from.] - T(3) = 9 + max{T(2), T(1)+1} = 9 + 17 = 26 <= 27 = 10 * 1 + 17 = 10 lg(3-1) + 17. [Why do we need base case for T(3)? Because we need n > 3 in inductive step below.] Ind. Hyp.: Let n > 3 and suppose T(j) <= 10 lg(j-1) + 17 for all 2 <= j < n. Ind. Step: Since n > 3, T(n) = 9 + max{ T(ceil(n/2)), T(floor(n/2)) + 1 }, <= 9 + max{ 10 lg(ceil(n/2)-1) + 17, 10 lg(floor(n/2)-1) + 17 + 1 } (by IH, since 2 <= floor(n/2) <= ceil(n/2) < n for n > 3) <= 9 + 10 lg((n-1)/2) + 17 + 1 (since floor(n/2)-1 <= ceil(n/2)-1 <= (n+1)/2-1 = (n-1)/2) <= 10 + 10 (lg(n-1) - lg(2)) + 17 <= 10 lg(n-1) + 17. Hence, by induction, T(n) <= 10 lg(n-1) + 17 for all n >= 2, i.e., T(n) (- O(lg n). Therefore, T(n) (- \Theta(log n). MergeSort(A,b,e): if b == e: return m = (b + e) / 2 MergeSort(A,b,m) MergeSort(A,m+1,e) # merge sorted A[b..m] and A[m+1..e] back into A[b..e] for i in [b..e]: B[c] = A[c] c = b d = m+1 for i in [b..e]: if d > e or (c <= m and B[c] < B[d]): A[i] = B[c] c += 1 else: # d <= e and (c > m or B[c] >= B[d]) A[i] = B[d] d += 1 - Recall: for i in [a..b]: BODY equivalent to: i = a while i <= b: BODY i += 1 with runtime 3 + (b - a + 1) * (3 + B) where B = runtime of BODY (counting 'i += 1' as a single "increment" operation). - Hence, worst-case runtime T(n) satisfies: { 3 if n = 1, T(n) = { { T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 if n > 1, where 21 n + 15 is the runtime of merging and additional operations outside the recursive calls. --------------------------------------------------------------------------- The following was not covered in lectures; it is included here for your reference, in case you were wondering exactly how we got the recurrence above. - Basic properties of floor() and ceil() that you can rely on: For all x (- R, k (- Z, . floor(-x) = -ceil(x), ceil(-x) = -floor(x) . floor(x+k) = floor(x)+k, ceil(x+k) = ceil(x)+k . if k is even, floor(k/2) = k/2 = ceil(k/2); if k is odd, floor(k/2) = (k-1)/2, ceil(k/2) = (k+1)/2 Exercise: based on these facts, you should have no trouble proving that for all k (- Z, floor((k+1)/2) = ceil(k/2). - The size of the input for the first recursive call is m+1-b. Lemma: floor((b+e)/2)-b+1 = ceil((e+1-b)/2). Proof: floor((b+e)/2)-b+1 = floor((b+e)/2-b+1) = floor((b+e-2b+2)/2) = floor((e+1-b+1)/2) = ceil((e+1-b)/2) So as a function of n = e+1-b, the size of the input for the first recursive call is ceil(n/2). - The size of the input for the second recursive call is e+1-(m+1) = e-m. Lemma: e-floor((b+e)/2) = floor((e+1-b)/2). Proof: e-floor((b+e)/2) = e+ceil(-(b+e)/2) = ceil(e-(b+e)/2) = ceil((2e-b-e)/2) = ceil((e-b)/2) = floor((e+1-b)/2) So as a function of n = e+1-b, the size of the input for the second recursive call is floor(n/2). --------------------------------------------------------------------------- - Repeated substitution: T(n) = 2 T(n/2) + 21 n + 15 = 2 (2 T(n/4) + 21(n/2) + 15) + 21 n + 15 = 4 T(n/4) + 2 * 21 n + 3 * 15 = 4 (2 T(n/8) + 21(n/4) + 15) + 2 * 21 n + 3 * 15 = 8 T(n/8) + 3 * 21 n + 7 * 15 after i steps = 2^i T(n/2^i) + 21 n i + 15 (2^i - 1) base reached when n/2^i = 1, i.e., i = lg n = 2^{lg n} T(n/n) + 21 n lg n + 15 (2^{lg n) - 1) = T(1) n + 21 n lg n + 15 n - 15 = 21 n lg n + 18 n - 15 - T(n) in Omega(n log n): Try T(n) >= 21 n lg n for all n >= 1. base: T(1) = 3 >= 0 = 21 (1) lg 1 I.H.: Let n > 1 and suppose T(j) >= 21 j lg j for 1 <= j < n. Step: Since n > 1, 1 <= floor(n/2) <= ceil(n/2) < n so T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 >= 21 ceil(n/2) lg ceil(n/2) + 21 floor(n/2) lg floor(n/2) + 21 n + 15 >= 21 (ceil(n/2) + floor(n/2)) lg floor(n/2) + 21 n + 15 >= 21 n (lg floor(n/2) + lg 2) + 15 >= 21 n lg(2 floor(n/2)) + 15 >= 21 n lg(2 (n-1)/2) + 15 >= 21 n lg(n-1) + 15 Problem! We want >= 21 n lg n. - Trick: ceil(n/2) + 1 >= floor(n/2) + 1 >= (n-1)/2 + 1 = (n+1)/2, so change statement to T(n) >= 21 n lg(n+1). Inductive step then becomes: T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 >= 21 ceil(n/2) lg(ceil(n/2)+1) + 21 floor(n/2) lg(floor(n/2)+1) + 21 n + 15 >= 21 (ceil(n/2) + floor(n/2)) lg((n+1)/2) + 21 n + 15 >= 21 n (lg((n+1)/2) + lg 2) + 15 >= 21 n lg(2 (n+1)/2) + 15 >= 21 n lg(n+1) + 15 >= 21 n lg(n+1) Exactly what we want! But new problem: base case doesn't work anymore T(1) = 3 NOT >= 21 = 21 (1) lg(1+1)... - Trick: inductive step has extra "+15" so try to change statement to T(n) >= 21 n lg(n+1) - 15. In inductive step, T(ceil(n/2)) contributes -15 and T(floor(n/2)) contributes -15: one of these cancels out +15 and other remains, as required. Base case? 21 (1) lg(1+1) - 15 = 21 - 15 = 6 NOT <= 3 = T(1) However, 21 (2) lg(2+1) - 15 <= 42 * 1.75 - 15 = 58.5 <= 63 = T(2) 21 (3) lg(3+1) - 15 = 126 - 15 = 111 <= 144 = T(3) We need both base cases because in ind. step, IH applies to floor(n/2) and ceil(n/2) only if 2 <= floor(n/2) <= ceil(n/2) < n, only if n >= 4. - Another possibility: instead of adding low-order term "-15", change constant and prove T(n) >= n lg(n+1) -- base case goes through and extra low-order terms don't matter. - T(n) in Omega(n log n): T(n) >= 21 n lg(n+1) - 15 for all n >= 2. base: T(2) = T(1) + T(1) + 21 * 2 + 15 = 3 + 3 + 42 + 15 = 63 >= 58.5 = 21 * 2 * 1.75 - 15 > 21 * 2 * lg 3 - 15 base: T(3) = T(2) + T(1) + 21 * 3 + 15 = 63 + 3 + 63 + 15 = 144 >= 111 = 21 * 3 * lg 4 - 15 I.H.: Let n > 3 and suppose T(j) >= 21 j lg(j+1) - 15 for 2 <= j < n. Step: Since n > 3, 2 <= floor(n/2) <= ceil(n/2) < n, and ceil(n/2)+1 >= floor(n/2)+1 >= (n-1)/2+1 = (n+1)/2, so T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 >= 21 ceil(n/2) lg(ceil(n/2)+1) - 15 + 21 floor(n/2) lg(floor(n/2)+1) - 15 + 21 n + 15 >= 21 (ceil(n/2) + floor(n/2)) lg((n+1)/2) + 21 n - 15 >= 21 n (lg(n+1) - lg 2 + 1) - 15 >= 21 n lg(n+1) - 15 - T(n) in O(n log n): Similar process of trial and error results in statement T(n) <= 21 n lg(n-1) + 39 n - 15 for all n >= 2 (-15 works as above, +39n required for base cases to add up). base: T(2) = 63 <= 63 = 78 - 15 = 21*2*0 + 39*2 - 15 T(3) = 144 <= 165 = 21*3*1 + 39*3 - 15 I.H.: Let n > 3 and suppose T(j) <= 21 j lg(j-1) + 39 n - 15 for 2 <= j < n. Step: Since n > 3, 2 <= floor(n/2) <= ceil(n/2) < n, and floor(n/2)-1 <= ceil(n/2)-1 <= (n+1)/2-1 = (n-1)/2, so T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 <= 21 ceil(n/2) lg(ceil(n/2)-1) + 39 ceil(n/2) - 15 + 21 floor(n/2) lg(floor(n/2)-1) + 39 floor(n/2) - 15 + 21 n + 15 <= 21 (ceil(n/2) + floor(n/2)) lg((n-1)/2) + 21 n + 39 (ceil(n/2) + floor(n/2)) - 15 <= 21 n (lg(n-1) - lg 2 + 1) + 39 n - 15 <= 21 n lg(n-1) + 39 n - 15 - Challenge: Prove that for all n >= 1, 21 n floor(lg n) + 18 n - 15 <= T(n) <= 21 n ceil(lg n) + 18 n - 15. Hint: In the inductive step, consider cases "n even" and "n odd", and use the fact that for all odd k >= 3, floor(lg k) = floor(lg(k-1)) and ceil(lg k) = ceil(lg(k+1)).