Chapter 4 Divide-and-Conquer 4.0.1 The steps of a divide-and-conquer algorithm: Divide: the problem into smaller subproblems Conquer: the subproblems recursively until arriving at small base cases which are solved directly Combine: the solutions to the subproblems into a solution to the original problem Two example problems: Maximum subarray problem: find the subarray of an array that has the largest sum. Matrix multiplication of n x n matrices: Strassen's algorithm runs in O(n^2.81) which beats the standard Theta(n^3) method. Recurrences The running time of a divide-and-conquer algorithm can often be described by a Recurrence: an equation or inequality that describes a function in terms of its value on smaller inputs. 4.0.2 For example, the running time of merge-sort: T(n) = / Theta(1) if n = 1 \ 2T(n/2) + Theta(n) if n > 1 whose solution was T(n) = Theta( n lg n ). We may have unequal divisions: e.g. 2/3 - 1/3 so T(n) = T(2n/3) + T(n/3) + Theta(n) Or even non-fraction divisions: for recursive linear search, T(n) = T(n-1) + Theta(1) Three methods for solving recurrences -- i.e. obtaining Theta or big-O bounds on T(n): 1) The substitution method: make a guess and then prove the guess is correct by induction. 2) The recursion-tree method: convert the recurrence into a tree of costs & sum them. 3) The "master method": gives bounds for recurrences of the form: T(n) = aT(n/b)+f(n) where a >= 1, b > 1, and f(n) is given. Sometimes the recurrences are inequalities: T(n) <= 2T(n/2) + Theta(n), in which case it is more appropriate to use O-notation. And if T(n) >= 2T(n/2) + Theta(n), Omega-notation would be more appropriate. Technicalities 4.0.3 1. Usually the running time T(n) is only defined for integer n. This is often ignored in practice. For instance, the precise recurrence for the run time of merge sort is: T(n) = / Theta(1) if n = 1 \ T(floor(n/2)) + T(ceiling(n/2)) + Theta(n) if n > 1 2. We usually don't give boundary conditions, since T(n) = Theta(1) for small n. Usually changing T(1) only changes the solution to the recurrence by a constant factor. So, for example, we usually just state the recurrence for the run time of merge sort simply as: T(n) = 2T(n/2) + Theta(n) Thus we usually omit floors, ceilings, and boundary conditions when we solve recurrences. Usually they don't matter, but we check to make sure that that is the case after we have obtained our solution. 4.1 The Maximum-subarray Problem 4.1.1 Suppose that you knew the price of a stock ahead of time. If you could only buy once and sell once later, when would you do it? Figure 4.1 shows the price and daily change of price of a stock over a period of time. Figure 4.2 shows that buying at the low point or selling at the high point may not be best. A Brute-force Solution The number of ways to choose a "buy" date and then a later "sell" day from n days is the binomial coefficient C(n,2) = n(n-1)/2, which is Theta(n^2). A Transformation To reduce the n^2 running time, we look at the input in terms of the daily differences in prices. Then the problem is equivalent to finding a subarray with the largest sum (there may be more than one such subarray). This itself doesn't help, since there are C(n,2) subarrays. We assume some entries are negative, since if all entries are positive, the entire array would be the easy solution. Part of Figure 4.3: 1 2 3 4 5 6 7 8 9 10 11 12 A: |13|-3|-25|20|-3|-16|-23|18|20|-7|12|-5|... \max subarr./ So A[8..11] is the maximum subarray; sum = 43 A Solution using Divide-and-Conquer 4.1.2 We want to find the maximum subarray of the array A[low..high]. Using divide-and-conquer, we divide it in half at mid: A[low..mid] and A[mid+1..high]. As shown in Figure 4.4, any subarray A[i..j], and the maximum subarray in particular, must: be entirely in A[low..mid] or be entirely in A[mid+1..high] or cross mid: low <= i <= mid < j <= high In the first two cases we can use recursion to find the max subarray. For the third case we can use Find-Max-Crossing-Subarray() which returns a 3-tuple (i,j,max-sum) where i is the index such that A[i..mid] is maximum, j is the index such that A[mid+1..j] is maximum, and max-sum is the maximum sum. This can be done in Theta(n) time where n = high - low + 1. 4.1.3 Find-Max-Crossing-Subarray(A, low, mid, high) 1 left-sum = -inf 2 sum = 0 3 for i = mid downto low 4 sum = sum + A[i] 5 if sum > left-sum 6 left-sum = sum 7 max-lt = i 8 right-sum = -inf 9 sum = 0 10 for j = mid+1 to high 11 sum = sum + A[j] 12 if sum > right-sum 13 right-sum = sum 14 max-rt = j 15 return (max-lt,max-rt,left-sum + right-sum) Then Find-Maximum-Subarray() uses recursion and Find-Max-Crossing-Subarray() to solve the problem, returning a 3-tuple (i,j,max-sum): Find-Maximum-Subarray(A, low, high) 1 If high == low 2 return ( low, high, A[low] ) 3 else mid = floor( (low + high)/2 ) 4 (left-low, left-high, left-sum) = Find-Maximum-Subarray(A, low, mid) 5 (right-low, right-high, right-sum) = Find-Maximum-Subarray(A, mid+1,high) 6 (cross-low, cross-high, cross-sum) = Find-Max-Crossing-Subarray(A,low,mid,high) 7 if left-sum >= right-sum and left-sum >= cross-sum 8 return (left-low, left-high, left-sum) 9 elseif right-sum >= left-sum and right-sum >= cross-sum 10 return (right-low,right-high,right-sum) 11 else return(cross-low,cross-high,cross-sum) 4.1.4 Analyzing the divide-and-conquer algorithm To analyze the algorithm we first note that the base case n = 1 takes constant time, so: T(1) = Theta(1) In the recursive case n > 1, for simplicity we assume n is a power of 2. Now lines 1, 3, and 7-11 take constant time. The recursive calls in lines 4 and 5 take a total of 2T(n/2) time. And as we noted line 6 takes Theta(n) time, so the total time is: T(n) = Theta(1) + 2T(n/2) + Theta(n) = 2T(n/2) + Theta(n) which gives the recurrence: T(n) = / Theta(1) if n = 1 \ 2T(n/2) + Theta(n) if n > 1 the same as for Merge-Sort, so the solution is T(n) = Theta(n lg n). However, Exercise 4.1-5 indicates that there is a Theta(n) solution. 4.2 Strassen's Algorithm for matrix 4.2.1 Multiplication To multiply two square nxn matrices A = (a_ij) and B = (b_ij), A*B = C = (c_ij) where n c_ij = Sum a_ik * b_kj which gives: k=1 SQUARE-MATRIX-MULTIPLY(A,B) 1 n = A.rows 2 let C be a new n x n matrix 3 for i = 1 to n 4 for j = 1 to n 5 c_ij = 0 6 for k = 1 to n 7 c_ij = c_ij + a_ik x b_kj 8 return C This is clearly a Theta(n^3) algorithm. About 40 years ago Strassen devised a Theta(n^lg(7)) algorithm ( 2.80 < lg(7) < 2.81 ). A Simple Divide-and-Conquer Algorithm For simplicity assume n is a power of 2. We then subdivide A, B, and C into four n/2 x n/2 submatrices: A = | A_11 A_12 | B = | B_11 B_12 | | A_21 A_22 | | B_21 B_22 | C = | C_11 C_12 | | C_21 C_22 | 4.2.2 then looking at the 4 submatrices of C = A * B C_11 = A_11 * B_11 + A_12 * B_21 C_12 = A_11 * B_12 + A_22 * B_22 C_21 = A_21 * B_11 + A_12 * B_21 C_22 = A_21 * B_12 + A_22 * B_22 This leads to the divide-and-conquer algorithm MATRIX-MULTIPLY-RECURSE(A,B) 1 n = A.rows 2 let C be a new n x n matrix 3 if n == 1 4 c_11 = a_11 * b_11 5 else partition A, B, and C as 4 submatrices 6 C_11 = MATRIX-MULTIPLY-RECURSE(A_11,B_11) + MATRIX-MULTIPLY-RECURSE(A_12,B_21) 7 C_12 = MATRIX-MULTIPLY-RECURSE(A_11,B_12) + MATRIX-MULTIPLY-RECURSE(A_12,B_22) 8 C_21 = MATRIX-MULTIPLY-RECURSE(A_21,B_11) + MATRIX-MULTIPLY-RECURSE(A_22,B_21) 9 C_22 = MATRIX-MULTIPLY-RECURSE(A_21,B_12) + MATRIX-MULTIPLY-RECURSE(A_22,B_22) 10 return C 4.2.3 Problem: line 5 seems to require creating 12 submatrices. Cure: use index calculations instead - i.e. use the existing matrices, so line 5 only take Theta(1) time. The base case, n = 1, takes Theta(1) time: T(1) = Theta(1) In the recursive case, there are 8 recursive calls to problems of size n/2, for 8T(n/2) time. There are also 4 matrix additions of (n/2)^2 entries for Theta(n^2) time. Thus the cost in the recursive case is: T(n) = Theta(1) + 8T(n/2) + Theta(n^2) = 8T(n/2) + Theta(n^2) As we will see, this recurrence has solution T(n) = Theta(n^3), no faster than the standard method. Strassen's method 4.2.4 The efficiency of Strassen's method is due to the reduction of the branching factor from 8 to 7 in the recursion tree. It trades one matrix multiplication for matrix additions, which take less time. It has four steps: 1. Divide A, B, and C into n/2 x n/2 matrices as in MATRIX-MULTIPLY-RECURSE(), using index calculations, thus taking Theta(1) time. 2. Create 10 n/2 x n/2 matrices S_1, S_2, ... S_10 each of which is a sum or difference of submatrices of A and B created in step 1. This takes Theta(n^2) time. 3. Recursively compute seven n/2 x n/2 matrix products P_1, P-2, ... P_7 of n/2 x n/2 matrices from steps 1 and 2, in 7T(n/2) time 4. Compute submatrices C_11, C_12, C_21, and C_22 by adding and subtracting various P_i matrices. This takes Theta(n^2) time. In the base case, when n = 1, we do a scalar multiplication as before, so T(1) = Theta(1). For n > 1, steps 1, 2, and 4 take a total of Theta(n^2) time, and step 3 takes 7T(n/2) time giving the recurrence for Strassen's method: T(n) = / Theta(1) if n = 1 4.2.5 \ 7T(n/2) + Theta(n^2) if n > 1 By the Master Method of Section 4.5, this recurrence has solution T(n) = Theta(n^lg(7)). Here are the details of steps 2-4: Step 2: S_1 = B_12 - B_22 S_2 = A_11 + A_12 S_3 = A_21 + A_22 S_4 = B_21 - B_11 S_5 = A_21 + A_22 S_6 = B_11 + B_22 S_7 = A_12 - A_22 S_8 = B_21 + B_22 S_9 = A_11 - A_21 S_10 = B_11 + B_12 Step 3: P_1 = A_11 * S_1 ( = A_11*B_12 - A_11*B_22 ) P_2 = S_2 * B_22 ( = A_11*B_22 + A_12*B_22 ) P_3 = S_3 * B_11 ( = A_21*B_11 + A_22*B_11 ) P_4 = A_22 * S_4 ( = A_22*B_21 - A_22*B_11 ) P_5 = S_5 * S_6 ( = A_11*B_11 + A_11*B_22 + A_22*B_11 + A_22*B_22 ) P_6 = S_7 * S_8 ( = A_12*B_21 + A_12*B_22 - A_22*B_21 - A_22*B_22 ) P_7 = S_9 * S_10 ( = A_11*B_11 + A_11*B_12 - A_21*B_11 - A_21*B_12 ) Step 4: 4.2.6 C_11 = P_5 + P_4 - P_2 + P_6 ( = A_11*B_11 + A_11*B_22 + A_22*B_11 + A_22*B_22 + + A_22*B_21 - A_22*B_11 - A_11*B_22 - A_12*B_22 + A_12*B_21 + A_12*B_22 + A_22*B_21 + A_22*B_22 = A_11*B_11 + A_12*B_21 as desired ) C_12 = P_1 + P_2 ( = A_11*B_12 - A_11*B_22 + A_11*B_22 + A_12*B_22 = A_11*B_12 + A_12*B_22 as desired ) C_21 = P_3 + P_4 ( = A_21*B_11 + A_22*B_11 + A_22*B_21 - A_22*B_11 = A_21*B_11 + A_22*B_21 as desired ) C_22 = P_5 + P_1 - P_3 - P_7 ( = A_11*B_11 + A_11*B_22 + A_22*B_11 + A_22*B_22 + A_11*B_12 - A_11*B_22 - A_21*B_11 + A_22*B_11 - A_11*B_11 - A_11*B_12 - A_22*B_11 - A_22*B_12 = A_22*B_22 + A_21*B_12 as desired ) Step 4 uses 8 matrix additions/subtractions, for Theta(n^2) time. Exercise 4.2-2 asks for the pseudocode for Strassen's algorithm. 4.3 The substitution method 4.3.1 The substitution method involves two steps: 1. Guess the form of the solution. 2. Use mathematical induction to find the constants and show that the solution works. This method can establish upper or lower bounds or both on a recurrence. For example, we determine an upper bound on the recurrence: T(n) = 2T(floor(n/2)) + n (4.19) 1. We guess the solution is T(n) = O(n lg n), so we need to prove 2. T(n) <= cn lg(n) for an appropriate choice of c. We start by assuming this holds for floor(n/2), i.e. that T(floor(n/2)) <= cfloor(n/2) lg(floor(n/2)) Substituting this into (4.19) above gives: T(n) <= 2[cfloor(n/2) lg(floor(n/2))] + n <= cn lg(n/2) + n = cn lg(n) - cn lg(2) + n = cn lg(n) - cn + n <= cn lg(n) where the last step holds if c >= 1. To complete the inductive proof, we need to pick c so that the inequality holds for the boundary case too. Now T(1) > 0, but lg(1) is 0, which seems to be a problem. But we only need T(n) <= cn lg(n) for 4.3.2 n >= n0, for some n0 > 0. We pick n0 = 2 and c with T(2) <= c 2 lg(2) and T(3) <= c 3 lg(3) to establish the base cases n = 2 and n = 3. Since T(2) = 2T(floor(2/2)) + 2 = 2T(1) + 2, and T(3) = 2T(floor(3/2)) + 3 = 2T(1) + 3, then letting c = maximum of (2T(1)+2)/(2lg2) and (2T(1)+3)/(3lg3) will work. The text shows that if T(1) = 1, then c = 2. Making a good guess Guessing is an art. But here are two methods: 1. If a recurrence is similar to one you have solved before, the solution to the new recurrence will be similar to the old one. Consider for example the recurrence: T(n) = 2T(floor(n/2) + 17) + n which is similar to (4.19) above. So you might correctly guess that T(n) = O(n lg n), which can be proved by the substitution method. 2. Guess easy upper and lower bounds, such as T(n) = Omega(n) and T(n) = O(n^2) for (4.19), and then improve them until they converge to the asymptotically tight T(n) = Theta(n lg n). Subtleties 4.3.3 A trick may be needed to make the solution go through -- subtracting a lower order term for example. Consider the recurrence: T(n) = T(floor(n/2)) + T(ceiling(n/2)) + 1 It is natural to guess the solution is O(n). But trying to prove T(n) <= cn leads to: T(n) <= c floor(n/2) + c ceiling(n/2) + 1 = c n + 1 which does not give T(n) <= c n for any choice of c. Now T(n) = O(n) is correct, but to show it, we have to guess that T(n) <= c n - b, so: T(n) <= c floor(n/2) - b + c ceiling(n/2) - b + 1 = c n - 2 b + 1 <= c n - b if we let b >= 1. As before, we must also choose c to handle the boundary conditions. Avoiding pitfalls It is easy to make mistakes. For example, we can "prove" T(n) = O(n) for (4.4) by guessing T(n) <= cn and then arguing: T(n) <= 2 c floor(n/2) + n <= c n + n = O(n) <=== Wrong!! The error is that we haven't proved the exact form of the inductive hypothesis: T(n) <= cn. Changing variables 4.3.4 Sometimes algebraic manipulation can make an unknown recurrence look like one you have solved. For example, consider the recurrence: T(n) = 2T(floor(sqrt(n))) + lg n Ignoring round-to-integer problems, we let m = lg(n), so that n = 2^m, and we get: T(2^m) = 2T(2^(m/2)) + m And now, letting S(m) = T(2^m), we have: S(m) = 2S(m/2) + m which we suspect (and can show) has the solution S(m) = O(m lg m). Changing back to T(n), T(n) = T(2^m) = S(m) = O(m lg m) = O(lg n lg lg n). 4.4 The recursion-tree method 4.4.1 In a recursion tree, each node represents the cost of a recursive call of the algorithm somewhere in its recursive invocation, but ignoring the cost of further recursive calls from within it, which are accounted for by "child" costs of that node. Once the costs of each node have been determined, the costs of the nodes at each level are summed, and then the per-level costs are summed to get the final answer. Recursion trees are especially useful in finding the running time of a divide-and-conquer algorithm Recursion trees are often used to generate good guesses, which can then be verified by the substitution method. Used in this way, we can tolerate a bit of "sloppiness" in the calculations. Or, by being careful, we can turn the guess into a proof (which is done in Section 4.6 when proving the master method). We first use a recursion tree to guess a solution to the recurrence: T(n) = 3T(floor(n/4)) + Theta(n^2) We want to find an upper bound for T(n). We ignore the floor function and non-integer values of n/4, so we can change the recurrence to: T(n) = 3T(n/4) + cn^2 for some c > 0. Figure 4.5, page 89, shows the construction of the recursion tree for this recurrence. Figure 4.5(b) shows one expansion of 4.4.2 T(n) into the non-recursive cost, cn^2, of the root and the three recursive costs, T(n/4), of its children: cn^2 / | \ / | \ / | \ / | \ / | \ T(n/4) T(n/4) T(n/4) Figure 4.5(c) shows the expansion of T(n) one level deeper, with each of the T(n/4) costs replaced by the non-recursive cost, c(n/4)^2, and the three recursive costs, T(n/16), of its children. We continue this process until we reach the boundary condition, a subproblem of size 1. If this occurs at depth i, n/4^i = 1, i.e. i = log_4(n). Thus the tree has log_4(n) + 1 levels: 0, 1, 2, ..., log_4(n), as is shown in Figure 4.5(d). To get the per-level cost, we note that the non-recursive cost at each level decreases by a factor of 4 and there are 3 times as many of them as in the previous level. So node cost at level i is c(n/4^i)^2 and there are 3^i such costs for a total of (3/16)^i cn^2. At the last level of depth log_4(n), there are 3^log_4(n) = n^log_4(3) nodes of cost T(1) for a total cost of Theta(n^log_4(3)). Thus the total cost of all levels is: 4.4.3 log_4(n)-1 T(n) = Sum ( (3/16)^i cn^2 ) + i=0 Theta(n^log_4(3)) inf < Sum( (3/16)^i cn^2) + Theta(n^log_4(3)) i=0 = 1/(1 - 3/16)cn^2 + Theta(n^log_4(3)) = 16/13 cn^2 + Theta(n^log_4(3)) = O(n^2) Which gives us our guess. Note that 13/13 of the 16/13 of the cn^2 term comes from the root and so T(n) = Omega(n^2), since it must take at least as much time to run as for the root. We now use the substitution method to verify our guess, T(n) = O(n^2) as an upper bound for the original recurrence: T(n) = 3T(floor(n/4)) + Theta(n^2) We need to show T(n) <= dn^2 for some d > 0. Letting c be as before, we have: T(n) <= 3T(floor(n/4)) + cn^2 <= 3d(floor(n/4))^2 + cn^2 by ind. hyp. <= 3d(n/4)^2 + cn^2 = (3/16)dn^2 + cn^2 <= dn^2 if we choose d >= (16/13)c For a second example we show 4.4.4 T(n) = O(n lg n) for the recurrence: T(n) = T(n/3) + T(2n/3) + O(n), i.e. T(n) = T(n/3) + T(2n/3) + cn. Figure 4.6 (page 91) shows the recurrence tree Cost cn cn ______/ \_____ / \ c(n/3) c(2n/3) cn / \ / \ / \ / \ c(n/9) c(2n/9) c(2n/9) c(4n/9) cn / \ / \ / \ / \ / \ / \ / \ / \ . . ________________ Total: O(n lg n) The longest path is down the right side, and stops when n(2/3)^k = 1, i.e. k = log n 3/2 So the total cost seems to be bounded by O(cn log_1.5(n)) = O(n lg n). But there is a problem: we have not accounted for the leaves. For a complete binary tree of height log_1.5(n), there would be 2^log_1.5(n) = n^log_1.5(2) leaves, each having a constant cost, for a total of Theta(n^log_1.5(2)) which is omega(n lg n). However, branches of the tree "die out" faster on the left than on the right, so there are < n^log_1.5(2) leaves. 4.4.5 It turns out that we don't have to do an exact accounting of all nodes, since we can prove T(n) = O(n lg n) by the substitution method by showing that T(n) <= dn lg(n), for some d > 0, as follows: T(n) = T(n/3) + T(2n/3) + cn <= d(n/3)lg(n/3) + d(2n/3)lg(2n/3) + cn by ind. hyp. = (d(n/3)lg(n) - d(n/3)lg(3)) + (d(2n/3)lg(n) - d(2n/3)lg(3/2)) + cn = dnlg(n) - d((n/3)lg(3) + (2n/3)lg(3/2)) + cn = dnlg(n) - d((n/3)lg(3)+(2n/3)lg(3)-(2n/3)lg(2)) + cn = dnlg(n) - dn(lg(3) -2/3) +cn <= dnlg(n) if we choose d >= c/(lg(3) -2/3). 4.5 The master method 4.5.1 The master method solves recurrences of the form: T(n) = aT(n/b)+f(n) for a >= 1 & b > 1. Such a recurrence describes the running time of an algorithm that divides a problem of size n into a subproblems of size n/b. The a subproblems are each solved in time T(n/b) and f(n) is the cost of dividing the problem and combining the solutions of the subproblems. If we want to be precise, n/b may not be an integer, but replacing it with floor(n/b) or ceiling(n/b) doesn't affect the asymptotic behavior of the solution, so we omit the floor and ceiling functions for convenience. The master theorem Theorem 4.1 (Master theorem) Let a >= 1 and b > 1, f(n) asymptotically positive, and let T(n) be defined by: T(n) = aT(n/b) + f(n) where n/b can be interpreted to be either floor(n/b) or ceiling(n/b). Then T(n) can be bounded asymptotically as follows: 1. If f(n) = O(n^(log_b(a)-epsilon)) for some epsilon > 0, then T(n) = Theta( n^log_b(a) ) 2. If f(n) = Theta( n^log_b(a) ), then T(n) = Theta( n^log_b(a) lg(n) ) 3. If f(n) = Omega(n^(log_b(a)+epsilon)) for some epsilon > 0, and if af(n/b) <= cf(n) for some c < 1 and all n > n0 for some n0 > 0, then T(n) = Theta( f(n) ) 4.5.2 In this theorem, f(n) is compared with n^log_b(a). In case 1, f(n) is polynomially smaller (by a factor of n^epsilon), then the recursive calls dominate, and the solution is T(n) = Theta( n^log_b(a) ). In case 3, f(n) is polynomially larger and satisfies the "regularity" condition af(n/b) <= cf(n) for some c < 1 and all n > n0 for some n0 > 0, then f(n) dominates and T(n) = Theta(f(n)) is the solution. Most of the polynomially bounded functions we consider satisfy this regularity condition. In case 2, f(n) and n^log_b(a) are the same size, which adds a lg(n) factor, so the solution is T(n) = Theta( n^log_b(a) lg(n) ). Note that these three cases don't cover all the possibilities. There is a "gap" between cases 1 and 2 when f(n) is smaller than n^log_b(a), but not polynomially smaller. There is a similar gap between cases 2 and 3 when f(n) is greater than n^log_b(a), but not polynomially greater. And even when f(n) is polynomially greater than n^log_b(a), the master theorem cannot be applied if f(n) does not satisfy the regularity condition. Using the master method 4.5.3 As a first example, let T(n) = 9T(n/3) + n, so a = 9, b = 3, f(n) = n, and n^log_b(a) = n^log_3(9) = n^2. The master theorem's case 1 applies since f(n) = O(n^log_3(9)-epsilon), where epsilon = 1 for instance, so that T(n) = Theta(n^log_3(9)) = Theta(n^2). As a second example, let T(n) = T(2n/3) + 1, so a = 1, b = 3/2, f(n) = 1, and n^log_b(a) = n^log_3/2(1) = n^0 = 1. The master theorem's case 2 applies, since f(n) = Theta(n^log_b(a)) = Theta(1), so T(n) = Theta( 1 lg(n) ). As a third example, let T(n) = 3T(n/4)+nlg(n) so a = 3, b = 4, f(n) = nlg(n), & n^log_b(a) = n^log_4(3) = O(n^0.793). Case 3 applies since f(n) = Omega(n^log_4(3)+epsilon), epsilon being about 0.2, and af(n/b) = 3(n/4)lg(n/4) <= (3/4)nlg(n) = cf(n) for c = 3/4. Thus the solution is T(n) = Theta( n lg(n) ). Case 3 _seems_ to apply to the recurrence T(n) = 2T(n/2) + nlg(n), which has the form a = 2, b = 2, f(n) = nlg(n), and n^log_b(a) = n^log_2(2) = n^1 = n, since f(n) = nlg(n) is asymptotically larger than n^log_b(a) = n. But it is not _polynomially_ larger, since f(n)/n^log_b(a) = nlg(n)/n = lg(n) is asymptotically less than n^epsilon for any positive constant epsilon, and so falls into the gap between cases 2 and 3; Exercise 4.6-2 shows the solution is T(n) = Theta(n lg^2(n)). We use the master method to solve 4.5.4 recurrences in Sections 4.1 and 4.2. T(n) = 2T(n/2) + Theta(n) is the recurrence for both merge sort and the maximum subarray problem. Here a = b = 2 and f(n) = Theta(n). Now n^log_b(a) = n^lg_2(2) = n, so case 2 applies and T(n) = Theta(n lg n). T(n) = 8T(n/2) + Theta(n^2) is the recurrence for recursive matrix multiply. Here a = 8, b = 2, and f(n) = n^2. Now n^log_b(a) = n^log_2(8) = n^3, which is polynomially larger than n^2 (i.e. f(n) = O(n^(3-epsilon) for epsilon = 1), so case 1 applies and T(n) = Theta(n^3). T(n) = 7T(n/2) + Theta(n^2) is the recurrence for Strassen's algorithm. Here a = 7, b = 2, and f(n) = n^2. Now n^log_b(a) = n^log_2(7), 2.80 < lg(7) < 2.81, and f(n) = O( n^(log_2(7)-epsilon) ) for epsilon = 0.8, so again so case 1 applies and T(n) = Theta( n^lg(7) ).