=========================================================================== CSC 236 Lecture Summary for Week 5 Fall 2007 =========================================================================== MergeSort(A,b,e): if b == e: return m = (b + e) / 2 MergeSort(A,b,m) MergeSort(A,m+1,e) # merge sorted A[b..m] and A[m+1..e] back into A[b..e] for i in [b..e]: B[c] = A[c] c = b d = m+1 for i in [b..e]: if d > e or (c <= m and B[c] < B[d]): A[i] = B[c] c += 1 else: # d <= e and (c > m or B[c] >= B[d]) A[i] = B[d] d += 1 - Recall: for i in [a..b]: BODY equivalent to: i = a while i <= b: BODY i += 1 with runtime 3 + (b - a + 1) * (3 + B) where B = runtime of BODY (counting 'i += 1' as a single "increment" operation). - Hence, worst-case runtime T(n) satisfies: { 3 if n = 1, T(n) = { { T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 if n > 1, where 21 n + 15 is the runtime of merging and additional operations outside the recursive calls. --------------------------------------------------------------------------- The following was not covered in lectures; it is included here for your reference, in case you were wondering exactly how we got the recurrence above. - Basic properties of floor() and ceil() that you can rely on: For all x in R, k in Z, . floor(-x) = -ceil(x), ceil(-x) = -floor(x) . floor(x+k) = floor(x)+k, ceil(x+k) = ceil(x)+k . if k is even, floor(k/2) = k/2 = ceil(k/2); if k is odd, floor(k/2) = (k-1)/2, ceil(k/2) = (k+1)/2 Exercise: based on these facts, you should have no trouble proving that for all k in Z, floor((k+1)/2) = ceil(k/2). - The size of the input for the first recursive call is m-b+1. Lemma: floor((b+e)/2)-b+1 = ceil((e-b+1)/2). Proof: floor((b+e)/2)-b+1 = floor((b+e)/2-b+1) = floor((b+e-2b+2)/2) = floor((e-b+1+1)/2) = ceil((e-b+1)/2) So as a function of n = e-b+1, the size of the input for the first recursive call is ceil(n/2). - The size of the input for the second recursive call is e-(m+1)+1 = e-m. Lemma: e-floor((b+e)/2) = floor((e-b+1)/2). Proof: e-floor((b+e)/2) = e+ceil(-(b+e)/2) = ceil(e-(b+e)/2) = ceil((2e-b-e)/2) = ceil((e-b)/2) = floor((e-b+1)/2) So as a function of n = e-b+1, the size of the input for the second recursive call is floor(n/2). --------------------------------------------------------------------------- - Repeated substitution: T(n) = 2 T(n/2) + 21 n + 15 = 2 (2 T(n/4) + 21(n/2) + 15) + 21 n + 15 = 4 T(n/4) + 2 * 21 n + 3 * 15 = 4 (2 T(n/8) + 21(n/4) + 15) + 2 * 21 n + 3 * 15 = 8 T(n/8) + 3 * 21 n + 7 * 15 after i steps = 2^i T(n/2^i) + 21 n i + 15 (2^i - 1) base reached when n/2^i = 1, i.e., i = lg n = 2^{lg n} T(n/n) + 21 n lg n + 15 (2^{lg n) - 1) = T(1) n + 21 n lg n + 15 n - 15 = 21 n lg n + 18 n - 15 - T(n) in Omega(n log n): Try T(n) >= 21 n lg n for all n >= 1. base: T(1) = 3 >= 0 = 21 (1) lg 1 I.H.: Let n > 1 and suppose T(j) >= 21 j lg j for 1 <= j < n. Step: Since n > 1, 1 <= floor(n/2) <= ceil(n/2) < n so T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 >= 21 ceil(n/2) lg ceil(n/2) + 21 floor(n/2) lg floor(n/2) + 21 n + 15 >= 21 (ceil(n/2) + floor(n/2)) lg floor(n/2) + 21 n + 15 >= 21 n (lg floor(n/2) + lg 2) + 15 >= 21 n lg(2 floor(n/2)) + 15 >= 21 n lg(2 (n-1)/2) + 15 >= 21 n lg(n-1) + 15 Problem! We want >= 21 n lg n. - Trick: ceil(n/2) + 1 >= floor(n/2) + 1 >= (n-1)/2 + 1 = (n+1)/2, so change statement to T(n) >= 21 n lg(n+1). Inductive step then becomes: T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 >= 21 ceil(n/2) lg(ceil(n/2)+1) + 21 floor(n/2) lg(floor(n/2)+1) + 21 n + 15 >= 21 (ceil(n/2) + floor(n/2)) lg((n+1)/2) + 21 n + 15 >= 21 n (lg((n+1)/2) + lg 2) + 15 >= 21 n lg(2 (n+1)/2) + 15 >= 21 n lg(n+1) + 15 >= 21 n lg(n+1) Exactly what we want! But new problem: base case doesn't work anymore T(1) = 3 NOT >= 21 = 21 (1) lg(1+1)... - Trick: inductive step has extra "+15" so try to change statement to T(n) >= 21 n lg(n+1) - 15. In inductive step, T(ceil(n/2)) contributes -15 and T(floor(n/2)) contributes -15: one of these cancels out +15 and other remains, as required. Base case? 21 (1) lg(1+1) - 15 = 21 - 15 = 6 NOT <= 3 = T(1) However, 21 (2) lg(2+1) - 15 <= 42 * 1.75 - 15 = 58.5 <= 63 = T(2) 21 (3) lg(3+1) - 15 = 126 - 15 = 111 <= 144 = T(3) We need both base cases because in ind. step, IH applies to floor(n/2) and ceil(n/2) only if 2 <= floor(n/2) <= ceil(n/2) < n, only if n >= 4. - T(n) in Omega(n log n): T(n) >= 21 n lg(n+1) - 15 for all n >= 2. base: T(2) = T(1) + T(1) + 21 * 2 + 15 = 3 + 3 + 42 + 15 = 63 >= 58.5 = 21 * 2 * 1.75 - 15 > 21 * 2 * lg 3 - 15 base: T(3) = T(2) + T(1) + 21 * 3 + 15 = 63 + 3 + 63 + 15 = 144 >= 111 = 21 * 3 * lg 4 - 15 I.H.: Let n > 3 and suppose T(j) >= 21 j lg(j+1) - 15 for 2 <= j < n. Step: Since n > 3, 2 <= floor(n/2) <= ceil(n/2) < n, and ceil(n/2)+1 >= floor(n/2)+1 >= (n-1)/2+1 = (n+1)/2, so T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 >= 21 ceil(n/2) lg(ceil(n/2)+1) - 15 + 21 floor(n/2) lg(floor(n/2)+1) - 15 + 21 n + 15 >= 21 (ceil(n/2) + floor(n/2)) lg((n+1)/2) + 21 n - 15 >= 21 n (lg(n+1) - lg 2 + 1) - 15 >= 21 n lg(n+1) - 15 - T(n) in O(n log n): Similar process of trial and error results in statement T(n) <= 21 n lg(n-1) + 39 n - 15 for all n >= 2 (-15 works as above, +39n required for base cases to add up). base: T(2) = 63 <= 63 = 78 - 15 = 21*2*0 + 39*2 - 15 T(3) = 144 <= 165 = 21*3*1 + 39*3 - 15 I.H.: Let n > 3 and suppose T(j) <= 21 j lg(j-1) + 39 n - 15 for 2 <= j < n. Step: Since n > 3, 2 <= floor(n/2) <= ceil(n/2) < n, and floor(n/2)-1 <= ceil(n/2)-1 <= (n+1)/2-1 = (n-1)/2, so T(n) = T(ceil(n/2)) + T(floor(n/2)) + 21 n + 15 <= 21 ceil(n/2) lg(ceil(n/2)-1) + 39 ceil(n/2) - 15 + 21 floor(n/2) lg(floor(n/2)-1) + 39 floor(n/2) - 15 + 21 n + 15 <= 21 (ceil(n/2) + floor(n/2)) lg((n-1)/2) + 21 n + 39 (ceil(n/2) + floor(n/2)) - 15 <= 21 n (lg(n-1) - lg 2 + 1) + 39 n - 15 <= 21 n lg(n-1) + 39 n - 15 - Challenge: Prove that for all n >= 1, 21 n floor(lg n) + 18 n - 15 <= T(n) <= 21 n ceil(lg n) + 18 n - 15. Hint: In the inductive step, consider cases "n even" and "n odd", and use the fact that for all odd k >= 3, floor(lg k) = floor(lg(k-1)) and ceil(lg k) = ceil(lg(k+1)).