Chapter 4 Divide-and-Conquer 4.0.1
The steps of a divide-and-conquer algorithm:
Divide: the problem into smaller subproblems
Conquer: the subproblems recursively until
arriving at small base cases which are
solved directly
Combine: the solutions to the subproblems
into a solution to the original problem
Two example problems:
Maximum subarray problem: find the subarray
of an array that has the largest sum.
Matrix multiplication of n x n matrices:
Strassen's algorithm runs in O(n^2.81)
which beats the standard Theta(n^3) method.
Recurrences
The running time of a divide-and-conquer
algorithm can often be described by a
Recurrence: an equation or inequality that
describes a function in terms of its value
on smaller inputs.
4.0.2
For example, the running time of merge-sort:
T(n) = / Theta(1) if n = 1
\ 2T(n/2) + Theta(n) if n > 1
whose solution was T(n) = Theta( n lg n ).
We may have unequal divisions: e.g. 2/3 - 1/3
so T(n) = T(2n/3) + T(n/3) + Theta(n)
Or even non-fraction divisions: for recursive
linear search, T(n) = T(n-1) + Theta(1)
Three methods for solving recurrences -- i.e.
obtaining Theta or big-O bounds on T(n):
1) The substitution method: make a guess and
then prove the guess is correct by induction.
2) The recursion-tree method: convert the
recurrence into a tree of costs & sum them.
3) The "master method": gives bounds for
recurrences of the form: T(n) = aT(n/b)+f(n)
where a >= 1, b > 1, and f(n) is given.
Sometimes the recurrences are inequalities:
T(n) <= 2T(n/2) + Theta(n), in which case it
is more appropriate to use O-notation. And if
T(n) >= 2T(n/2) + Theta(n), Omega-notation
would be more appropriate.
Technicalities 4.0.3
1. Usually the running time T(n) is only
defined for integer n. This is often ignored
in practice. For instance, the precise
recurrence for the run time of merge sort is:
T(n) = / Theta(1) if n = 1
\ T(floor(n/2)) + T(ceiling(n/2)) +
Theta(n) if n > 1
2. We usually don't give boundary conditions,
since T(n) = Theta(1) for small n. Usually
changing T(1) only changes the solution to
the recurrence by a constant factor. So, for
example, we usually just state the recurrence
for the run time of merge sort simply as:
T(n) = 2T(n/2) + Theta(n)
Thus we usually omit floors, ceilings, and
boundary conditions when we solve recurrences.
Usually they don't matter, but we check to
make sure that that is the case after we have
obtained our solution.
4.1 The Maximum-subarray Problem 4.1.1
Suppose that you knew the price of a stock
ahead of time. If you could only buy once and
sell once later, when would you do it?
Figure 4.1 shows the price and daily change
of price of a stock over a period of time.
Figure 4.2 shows that buying at the low point
or selling at the high point may not be best.
A Brute-force Solution
The number of ways to choose a "buy" date
and then a later "sell" day from n days is
the binomial coefficient C(n,2) = n(n-1)/2,
which is Theta(n^2).
A Transformation
To reduce the n^2 running time, we look at
the input in terms of the daily differences in
prices. Then the problem is equivalent to
finding a subarray with the largest sum
(there may be more than one such subarray).
This itself doesn't help, since there are
C(n,2) subarrays. We assume some entries are
negative, since if all entries are positive,
the entire array would be the easy solution.
Part of Figure 4.3:
1 2 3 4 5 6 7 8 9 10 11 12
A: |13|-3|-25|20|-3|-16|-23|18|20|-7|12|-5|...
\max subarr./
So A[8..11] is the maximum subarray; sum = 43
A Solution using Divide-and-Conquer 4.1.2
We want to find the maximum subarray of the
array A[low..high]. Using divide-and-conquer,
we divide it in half at mid: A[low..mid] and
A[mid+1..high]. As shown in Figure 4.4, any
subarray A[i..j], and the maximum subarray in
particular, must:
be entirely in A[low..mid] or
be entirely in A[mid+1..high] or
cross mid: low <= i <= mid < j <= high
In the first two cases we can use recursion
to find the max subarray. For the third case
we can use Find-Max-Crossing-Subarray() which
returns a 3-tuple (i,j,max-sum) where i is the
index such that A[i..mid] is maximum, j is the
index such that A[mid+1..j] is maximum, and
max-sum is the maximum sum. This can be done
in Theta(n) time where n = high - low + 1.
4.1.3
Find-Max-Crossing-Subarray(A, low, mid, high)
1 left-sum = -inf
2 sum = 0
3 for i = mid downto low
4 sum = sum + A[i]
5 if sum > left-sum
6 left-sum = sum
7 max-lt = i
8 right-sum = -inf
9 sum = 0
10 for j = mid+1 to high
11 sum = sum + A[j]
12 if sum > right-sum
13 right-sum = sum
14 max-rt = j
15 return (max-lt,max-rt,left-sum + right-sum)
Then Find-Maximum-Subarray() uses recursion
and Find-Max-Crossing-Subarray() to solve the
problem, returning a 3-tuple (i,j,max-sum):
Find-Maximum-Subarray(A, low, high)
1 If high == low
2 return ( low, high, A[low] )
3 else mid = floor( (low + high)/2 )
4 (left-low, left-high, left-sum) =
Find-Maximum-Subarray(A, low, mid)
5 (right-low, right-high, right-sum) =
Find-Maximum-Subarray(A, mid+1,high)
6 (cross-low, cross-high, cross-sum) =
Find-Max-Crossing-Subarray(A,low,mid,high)
7 if left-sum >= right-sum and
left-sum >= cross-sum
8 return (left-low, left-high, left-sum)
9 elseif right-sum >= left-sum and
right-sum >= cross-sum
10 return (right-low,right-high,right-sum)
11 else return(cross-low,cross-high,cross-sum)
4.1.4
Analyzing the divide-and-conquer algorithm
To analyze the algorithm we first note that
the base case n = 1 takes constant time, so:
T(1) = Theta(1)
In the recursive case n > 1, for simplicity
we assume n is a power of 2. Now lines 1, 3,
and 7-11 take constant time. The recursive
calls in lines 4 and 5 take a total of 2T(n/2)
time. And as we noted line 6 takes Theta(n)
time, so the total time is:
T(n) = Theta(1) + 2T(n/2) + Theta(n)
= 2T(n/2) + Theta(n)
which gives the recurrence:
T(n) = / Theta(1) if n = 1
\ 2T(n/2) + Theta(n) if n > 1
the same as for Merge-Sort, so the solution is
T(n) = Theta(n lg n). However, Exercise 4.1-5
indicates that there is a Theta(n) solution.
4.2 Strassen's Algorithm for matrix 4.2.1
Multiplication
To multiply two square nxn matrices A = (a_ij)
and B = (b_ij), A*B = C = (c_ij) where
n
c_ij = Sum a_ik * b_kj which gives:
k=1
SQUARE-MATRIX-MULTIPLY(A,B)
1 n = A.rows
2 let C be a new n x n matrix
3 for i = 1 to n
4 for j = 1 to n
5 c_ij = 0
6 for k = 1 to n
7 c_ij = c_ij + a_ik x b_kj
8 return C
This is clearly a Theta(n^3) algorithm. About
40 years ago Strassen devised a Theta(n^lg(7))
algorithm ( 2.80 < lg(7) < 2.81 ).
A Simple Divide-and-Conquer Algorithm
For simplicity assume n is a power of 2. We
then subdivide A, B, and C into four
n/2 x n/2 submatrices:
A = | A_11 A_12 | B = | B_11 B_12 |
| A_21 A_22 | | B_21 B_22 |
C = | C_11 C_12 |
| C_21 C_22 |
4.2.2
then looking at the 4 submatrices of C = A * B
C_11 = A_11 * B_11 + A_12 * B_21
C_12 = A_11 * B_12 + A_22 * B_22
C_21 = A_21 * B_11 + A_12 * B_21
C_22 = A_21 * B_12 + A_22 * B_22
This leads to the divide-and-conquer algorithm
MATRIX-MULTIPLY-RECURSE(A,B)
1 n = A.rows
2 let C be a new n x n matrix
3 if n == 1
4 c_11 = a_11 * b_11
5 else partition A, B, and C as 4 submatrices
6 C_11 = MATRIX-MULTIPLY-RECURSE(A_11,B_11)
+ MATRIX-MULTIPLY-RECURSE(A_12,B_21)
7 C_12 = MATRIX-MULTIPLY-RECURSE(A_11,B_12)
+ MATRIX-MULTIPLY-RECURSE(A_12,B_22)
8 C_21 = MATRIX-MULTIPLY-RECURSE(A_21,B_11)
+ MATRIX-MULTIPLY-RECURSE(A_22,B_21)
9 C_22 = MATRIX-MULTIPLY-RECURSE(A_21,B_12)
+ MATRIX-MULTIPLY-RECURSE(A_22,B_22)
10 return C
4.2.3
Problem: line 5 seems to require creating 12
submatrices. Cure: use index calculations
instead - i.e. use the existing matrices, so
line 5 only take Theta(1) time.
The base case, n = 1, takes Theta(1) time:
T(1) = Theta(1)
In the recursive case, there are 8 recursive
calls to problems of size n/2, for 8T(n/2)
time. There are also 4 matrix additions of
(n/2)^2 entries for Theta(n^2) time. Thus
the cost in the recursive case is:
T(n) = Theta(1) + 8T(n/2) + Theta(n^2)
= 8T(n/2) + Theta(n^2)
As we will see, this recurrence has solution
T(n) = Theta(n^3), no faster than the standard
method.
Strassen's method 4.2.4
The efficiency of Strassen's method is due to
the reduction of the branching factor from 8
to 7 in the recursion tree. It trades one
matrix multiplication for matrix additions,
which take less time. It has four steps:
1. Divide A, B, and C into n/2 x n/2 matrices
as in MATRIX-MULTIPLY-RECURSE(), using index
calculations, thus taking Theta(1) time.
2. Create 10 n/2 x n/2 matrices S_1, S_2, ...
S_10 each of which is a sum or difference of
submatrices of A and B created in step 1.
This takes Theta(n^2) time.
3. Recursively compute seven n/2 x n/2 matrix
products P_1, P-2, ... P_7 of n/2 x n/2
matrices from steps 1 and 2, in 7T(n/2) time
4. Compute submatrices C_11, C_12, C_21, and
C_22 by adding and subtracting various P_i
matrices. This takes Theta(n^2) time.
In the base case, when n = 1, we do a scalar
multiplication as before, so T(1) = Theta(1).
For n > 1, steps 1, 2, and 4 take a total of
Theta(n^2) time, and step 3 takes 7T(n/2) time
giving the recurrence for Strassen's method:
T(n) = / Theta(1) if n = 1 4.2.5
\ 7T(n/2) + Theta(n^2) if n > 1
By the Master Method of Section 4.4, this
recurrence has solution T(n) = Theta(n^lg(7)).
Here are the details of steps 2-4:
Step 2:
S_1 = B_12 - B_22
S_2 = A_11 + A_12
S_3 = A_21 + A_22
S_4 = B_21 - B_11
S_5 = A_21 + A_22
S_6 = B_11 + B_22
S_7 = A_12 - A_22
S_8 = B_21 + B_22
S_9 = A_11 - A_21
S_10 = B_11 + B_12
Step 3:
P_1 = A_11 * S_1 ( = A_11*B_12 - A_11*B_22 )
P_2 = S_2 * B_22 ( = A_11*B_22 + A_12*B_22 )
P_3 = S_3 * B_11 ( = A_21*B_11 + A_22*B_11 )
P_4 = A_11 * S_1 ( = A_22*B_21 - A_22*B_11 )
P_5 = S_5 * S_6 ( = A_11*B_11 + A_11*B_22
+ A_22*B_11 + A_22*B_22 )
P_6 = S_7 * S_8 ( = A_12*B_21 + A_12*B_22
+ A_22*B_21 + A_22*B_22 )
P_7 = S_9 * S_10 ( = A_11*B_11 + A_11*B_12
+ A_22*B_11 + A_22*B_12 )
Step 4: 4.2.6
C_11 = P_5 + P_4 - P_2 + P_6 ( = A_11*B_11 +
A_11*B_22 + A_22*B_11 + A_22*B_22 +
+ A_22*B_21 - A_22*B_11
- A_11*B_22 - A_12*B_22 + A_12*B_21 +
A_12*B_22 + A_22*B_21 + A_22*B_22
= A_11*B_11 + A_12*B_21 as desired )
C_12 = P_1 + P_2 ( = A_11*B_12 - A_11*B_22
+ A_11*B_22 + A_12*B_22
= A_11*B_12 + A_12*B_22 as desired )
C_21 = P_3 + P_4 ( = A_21*B_11 + A_22*B_11
+ A_22*B_21 - A_22*B_11
= A_21*B_11 + A_22*B_21 as desired )
C_22 = P_5 + P_1 - P_3 - P_7 ( = A_11*B_11 +
A_11*B_22 + A_22*B_11 + A_22*B_22
+ A_11*B_12 - A_11*B_22
- A_21*B_11 + A_22*B_11 - A_11*B_11
- A_11*B_12 - A_22*B_11 - A_22*B_12
= A_22*B_22 + A_21*B_12 as desired )
Step 4 uses 8 matrix additions/subtractions,
for Theta(n^2) time. Exercise 4.2-2 asks for
the pseudocode for Strassen's algorithm.
4.3 The substitution method 4.3.1
The substitution method involves two steps:
1. Guess the form of the solution.
2. Use mathematical induction to find the
constants and show that the solution works.
This method can establish upper or lower
bounds or both on a recurrence. For example,
we determine an upper bound on the recurrence:
T(n) = 2T(floor(n/2)) + n (4.19)
1. We guess the solution is T(n) = O(n lg n),
so we need to prove 2. T(n) <= cn lg(n) for an
appropriate choice of c. We start by assuming
this holds for floor(n/2), i.e. that
T(floor(n/2)) <= cfloor(n/2) lg(floor(n/2))
Substituting this into (4.19) above gives:
T(n) <= 2[cfloor(n/2) lg(floor(n/2))] + n
<= cn lg(n/2) + n
= cn lg(n) - cn lg(2) + n
= cn lg(n) - cn + n
<= cn lg(n)
where the last step holds if c >= 1.
To complete the inductive proof, we need to
pick c so that the inequality holds for the
boundary case too. Now T(1) > 0, but lg(1) is
0, which seems to be a problem.
But we only need T(n) <= cn lg(n) for 4.3.2
n >= n0, for some n0 > 0. We pick n0 = 2 and
c with T(2) <= c 2 lg(2) and T(3) < c 3 lg(3)
to establish the base cases n = 2 and n = 3.
Since T(2) = 2T(floor(2/2)) + 2 = 2T(1) + 2,
and T(3) = 2T(floor(3/2)) + 3 = 2T(1) + 3,
then letting c = maximum of (2T(1)+2)/(2lg2)
and (2T(1)+3)/(3lg3) will work. The text
shows that if T(1) = 1, then c = 2.
Making a good guess
Guessing is an art. But here are two methods:
1. If a recurrence is similar to one you have
solved before, the solution to the new
recurrence will be similar to the old one.
Consider for example the recurrence:
T(n) = 2T(floor(n/2) + 17) + n
which is similar to (4.19) above. So you might
correctly guess that T(n) = O(n lg n), which
can be proved by the substitution method.
2. Guess easy upper and lower bounds, such as
T(n) = Omega(n) and T(n) = O(n^2) for (4.4),
and then improve them until they converge to
the asymptotically tight T(n) = Theta(n lg n).
Subtleties 4.3.3
A trick may be needed to make the solution go
through -- subtracting a lower order term for
example. Consider the recurrence:
T(n) = T(floor(n/2)) + T(ceiling(n/2)) + 1
It is natural to guess the solution is O(n).
But trying to prove T(n) <= cn leads to:
T(n) <= c floor(n/2) + c ceiling(n/2) + 1
= c n + 1
which does not give T(n) <= c n for any choice
of c. Now T(n) = O(n) is correct, but to show
it, we have to guess that T(n) <= c n - b, so:
T(n) <= c floor(n/2) - b + c ceiling(n/2) - b
+ 1
= c n - 2 b + 1
<= c n - b
if we let b >= 1. As before, we must also
choose c to handle the boundary conditions.
Avoiding pitfalls
It is easy to make mistakes. For example, we
can "prove" T(n) = O(n) for (4.4) by guessing
T(n) <= cn and then arguing:
T(n) <= 2 c floor(n/2) + n
<= c n + n
= O(n) <=== Wrong!!
The error is that we haven't proved the exact
form of the inductive hypothesis: T(n) <= cn.
Changing variables 4.3.4
Sometimes algebraic manipulation can make an
unknown recurrence look like one you have
solved. For example, consider the recurrence:
T(n) = 2T(floor(sqrt(n))) + lg n
Ignoring round-to-integer problems, we let
m = lg(n), so that n = 2^m, and we get:
T(2^m) = 2T(2^(m/2)) + m
And now, letting S(m) = T(2^m), we have:
S(m) = 2S(m/2) + m
which we suspect (and can show) has the
solution S(m) = O(m lg m). Changing back to
T(n), T(n) = T(2^m) = S(m) = O(m lg m) =
O(lg n lg lg n).
4.4 The recursion-tree method 4.4.1
In a recursion tree, each node represents the
cost of a recursive call of the algorithm
somewhere in its recursive invocation, but
ignoring the cost of further recursive calls
from within it, which are accounted for by
"child" costs of that node. Once the costs of
each node have been determined, the costs of
the nodes at each level are summed, and then
the per-level costs are summed to get the
final answer. Recursion trees are especially
useful in finding the running time of a
divide-and-conquer algorithm
Recursion trees are often used to generate
good guesses, which can then be verified by
the substitution method. Used in this way,
we can tolerate a bit of "sloppiness" in the
calculations. Or, by being careful, we can
turn the guess into a proof (which is done in
Section 4.4 when proving the master method).
We first use a recursion tree to guess a
solution to the recurrence:
T(n) = 3T(floor(n/4)) + Theta(n^2)
We want to find an upper bound for T(n). We
ignore the floor function and non-integer
values of n/4, so we can change the recurrence
to: T(n) = 3T(n/4) + cn^2 for some c > 0.
Figure 4.5, page 89, shows the construction of
the recursion tree for this recurrence.
Figure 4.5(b) shows one expansion of 4.4.2
T(n) into the non-recursive cost, cn^2, of the
root and the three recursive costs, T(n/4),
of its children:
cn^2
/ | \
/ | \
/ | \
/ | \
/ | \
T(n/4) T(n/4) T(n/4)
Figure 4.5(c) shows the expansion of T(n) one
level deeper, with each of the T(n/4) costs
replaced by the non-recursive cost, c(n/4)^2,
and the three recursive costs, T(n/16), of its
children. We continue this process until we
reach the boundary condition, a subproblem of
size 1. If this occurs at depth i, n/4^i = 1,
i.e. i = log_4(n). Thus the tree has
log_4(n) + 1 levels: 0, 1, 2, ..., log_4(n),
as is shown in Figure 4.5(d).
To get the per-level cost, we note that the
non-recursive cost at each level decreases by
a factor of 4 and there are 3 times as many of
them as in the previous level. So node cost
at level i is c(n/4^i)^2 and there are 3^i
such costs for a total of (3/16)^i cn^2. At
the last level of depth log_4(n), there are
3^log_4(n) = n^log_4(3) nodes of cost T(1) for
a total cost of Theta(n^log_4(3)).
Thus the total cost of all levels is: 4.4.3
log_4(n)-1
T(n) = Sum ( (3/16)^i cn^2 ) +
i=0 Theta(n^log_4(3))
inf
< Sum( (3/16)^i cn^2) + Theta(n^log_4(3))
i=0
= 1/(1 - 3/16)cn^2 + Theta(n^log_4(3))
= 16/13 cn^2 + Theta(n^log_4(3))
= O(n^2)
Which gives us our guess. Note that 13/13 of
the 16/13 of the cn^2 term comes from the root
and so T(n) = Omega(n^2), since it must take
at least as much time to run as for the root.
We now use the substitution method to verify
our guess, T(n) = O(n^2) as an upper bound for
the original recurrence:
T(n) = 3T(floor(n/4)) + Theta(n^2)
We need to show T(n) <= dn^2 for some d > 0.
Letting c be as before, we have:
T(n) <= 3T(floor(n/4)) + cn^2
<= 3d(floor(n/4))^2 + cn^2 by ind. hyp.
<= 3d(n/4)^2 + cn^2
= (3/16)dn^2 + cn^2
<= dn^2
if we choose d >= (16/13)c
For a second example we show 4.4.4
T(n) = O(n lg n) for the recurrence:
T(n) = T(n/3) + T(2n/3) + O(n), i.e.
T(n) = T(n/3) + T(2n/3) + cn.
Figure 4.6 (page 91) shows the recurrence tree
Cost
cn cn
______/ \_____
/ \
c(n/3) c(2n/3) cn
/ \ / \
/ \ / \
c(n/9) c(2n/9) c(2n/9) c(4n/9) cn
/ \ / \ / \ / \
/ \ / \ / \ / \
.
. ________________
Total: O(n lg n)
The longest path is down the right side, and
stops when n(2/3)^k = 1, i.e. k = log n
3/2
So the total cost seems to be bounded by
O(cn log_1.5(n)) = O(n lg n).
But there is a problem: we have not accounted
for the leaves. For a complete binary tree of
height log_1.5(n), there would be 2^log_1.5(n)
= n^log_1.5(2) leaves, each having a constant
cost, for a total of Theta(n^log_1.5(2)) which
is omega(n lg n). However, branches of the
tree "die out" faster on the left than on the
right, so there are < n^log_1.5(2) leaves.
4.4.5
It turns out that we don't have to do an
exact accounting of all nodes, since we can
prove T(n) = O(n lg n) by the substitution
method by showing that T(n) <= dn lg(n), for
some d > 0, as follows:
T(n) = T(n/3) + T(2n/3) + cn
<= d(n/3)lg(n/3) + d(2n/3)lg(2n/3) + cn
by ind. hyp.
= (d(n/3)lg(n) - d(n/3)lg(3))
+ (d(2n/3)lg(n) - d(2n/3)lg(3/2)) + cn
= dnlg(n) - d((n/3)lg(3) + (2n/3)lg(3/2))
+ cn
= dnlg(n)
- d((n/3)lg(3)+(2n/3)lg(3)-(2n/3)lg(2))
+ cn
= dnlg(n) - dn(lg(3) -2/3) +cn
<= dnlg(n)
if we choose d >= c/(lg(3) -2/3).
4.5 The master method 4.5.1
The master method solves recurrences of the
form: T(n) = aT(n/b)+f(n) for a >= 1 & b > 1.
Such a recurrence describes the running time
of an algorithm that divides a problem of
size n into a subproblems of size n/b. The a
subproblems are each solved in time T(n/b) and
f(n) is the cost of dividing the problem and
combining the solutions of the subproblems.
If we want to be precise, n/b may not be an
integer, but replacing it with floor(n/b) or
ceiling(n/b) doesn't affect the asymptotic
behavior of the solution, so we omit the floor
and ceiling functions for convenience.
The master theorem
Theorem 4.1 (Master theorem)
Let a >= 1 and b > 1, f(n) asymptotically
positive, and let T(n) be defined by:
T(n) = aT(n/b) + f(n)
where n/b can be interpreted to be either
floor(n/b) or ceiling(n/b). Then T(n) can be
bounded asymptotically as follows:
1. If f(n) = O(n^(log_b(a)-epsilon)) for some
epsilon > 0, then T(n) = Theta( n^log_b(a) )
2. If f(n) = Theta( n^log_b(a) ), then
T(n) = Theta( n^log_b(a) lg(n) )
3. If f(n) = Omega(n^(log_b(a)+epsilon)) for
some epsilon > 0, and if af(n/b) <= cf(n)
for some c < 1 and all n > n0 for some
n0 > 0, then T(n) = Theta( f(n) )
4.5.2
In this theorem, f(n) is compared with
n^log_b(a). In case 1, f(n) is polynomially
smaller (by a factor of n^epsilon), then the
recursive calls dominate, and the solution is
T(n) = Theta( n^log_b(a) ). In case 3, f(n)
is polynomially larger and satisfies the
"regularity" condition af(n/b) <= cf(n) for
some c < 1 and all n > n0 for some n0 > 0,
then f(n) dominates and T(n) = Theta(f(n)) is
the solution. Most of the polynomially
bounded functions we consider satisfy this
regularity condition. In case 2, f(n) and
n^log_b(a) are the same size, which adds a
lg(n) factor, so the solution is
T(n) = Theta( n^log_b(a) lg(n) ).
Note that these three cases don't cover all
the possibilities. There is a "gap" between
cases 1 and 2 when f(n) is smaller than
n^log_b(a), but not polynomially smaller.
There is a similar gap between cases 2 and 3
when f(n) is greater than n^log_b(a), but not
polynomially greater. And even when f(n) is
polynomially greater than n^log_b(a), the
master theorem cannot be applied if f(n) does
not satisfy the regularity condition.
Using the master method 4.5.3
As a first example, let T(n) = 9T(n/3) + n,
so a = 9, b = 3, f(n) = n, and n^log_b(a) =
n^log_3(9) = n^2. The master theorem's case 1
applies since f(n) = O(n^log_3(9)-epsilon),
where epsilon = 1 for instance, so that
T(n) = Theta(n^log_3(9)) = Theta(n^2).
As a second example, let T(n) = T(2n/3) + 1,
so a = 1, b = 3/2, f(n) = 1, and n^log_b(a) =
n^log_3/2(1) = n^0 = 1. The master theorem's
case 2 applies, since f(n) = Theta(n^log_b(a))
= Theta(1), so T(n) = Theta( 1 lg(n) ).
As a third example, let T(n) = 3T(n/4)+nlg(n)
so a = 3, b = 4, f(n) = nlg(n), & n^log_b(a) =
n^log_4(3) = O(n^0.793). Case 3 applies since
f(n) = Omega(n^log_4(3)+epsilon), epsilon
being about 0.2, and af(n/b) = 3(n/4)lg(n/4)
<= (3/4)nlg(n) = cf(n) for c = 3/4. Thus the
solution is T(n) = Theta( n lg(n) ).
Case 3 _seems_ to apply to the recurrence
T(n) = 2T(n/2) + nlg(n), which has the form
a = 2, b = 2, f(n) = nlg(n), and n^log_b(a) =
n^log_2(2) = n^1 = n, since f(n) = nlg(n) is
asymptotically larger than n^log_b(a) = n.
But it is not _polynomially_ larger, since
f(n)/n^log_b(a) = nlg(n)/n = lg(n) is
asymptotically less than n^epsilon for any
positive constant epsilon, and so falls into
the gap between cases 2 and 3; Exercise 4.6-2
shows the solution is T(n) = Theta(n lg^2(n)).
We use the master method to solve 4.5.4
recurrences in Sections 4.1 and 4.2.
T(n) = 2T(n/2) + Theta(n)
is the recurrence for both merge sort and the
maximum subarray problem. Here a = b = 2 and
f(n) = Theta(n). Now n^log_b(a) = n^lg_2(2) =
n, so case 2 applies and T(n) = Theta(n lg n).
T(n) = 8T(n/2) + Theta(n^2)
is the recurrence for recursive matrix
multiply. Here a = 8, b = 2, and f(n) = n^2.
Now n^log_b(a) = n^log_2(8) = n^3, which is
polynomially larger than n^2 (i.e. f(n) =
O(n^(3-epsilon) for epsilon = 1), so case 1
applies and T(n) = Theta(n^3).
T(n) = 7T(n/2) + Theta(n^2)
is the recurrence for Strassen's algorithm.
Here a = 7, b = 2, and f(n) = n^2. Now
n^log_b(a) = n^log_2(7), 2.80 < lg(7) < 2.81,
and f(n) = O( n^(log_2(7)-epsilon) ) for
epsilon = 0.8, so again so case 1 applies and
T(n) = Theta( n^lg(7) ).