Divide and Conquer Algorithms
F IND (A , key )
1 for i ← 1 to length (A)
2 do
3 if A [i] = key
4 then return TRUE
5 return FALSE 5 return FALSE
F IND (A , key )
F IND I N L IST (A , key )
1 for i ← 1 to length (A)
2 do
2 item ← first (A)
3 if A [i] = key
3 while item = last(A) 6
4 then return TRUE 4 do
5 return FALSE
5 if value (item) = key
6 then return TRUE
7 i ← i +1
8 item ← next (item)
9 return FALSE 9 return FALSE
F IND (A , key )
F IND I N L IST (A , key )
1 for i ← 1 to length (A)
2 do
2 item ← first (A)
3 if A [i] = key
3 while item = last(A) 6
4 then return TRUE 4 do
5 return FALSE
5 if value (item) = key
6 then return TRUE
7 i ← i +1
8 item ← next (item)
9 return FALSE The complexity of F IND and F IND I N L IST (with N = length(A)) is 9 return FALSE The complexity of F IND and F IND I N L IST (with N = length(A)) is
F IND (A , key )
F IND I N L IST (A , key )
1 for i ← 1 to length (A)
2 do
2 item ← first (A)
3 if A [i] = key
3 while item 6 = last(A)
4 then return TRUE 4 do
5 return FALSE
5 if value (item) = key
6 then return TRUE
7 i ← i +1
8 item ← next (item)
9 return FALSE The complexity of F IND and F IND I N L IST (with N = length(A)) is
T (N) = O(N)
1 for i ← 1 to length (A)
2 do
3 if ¬ F IND (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ F IND (A , B [i]) ∧¬ F IND (B[1 .. i − 1 ] , B [i])
8 then output B [i]
T (N) = ∑ T F IND (i)
i =1
1 for i ← 1 to length (A)
2 do
3 if ¬ F IND (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ F IND (A , B [i]) ∧¬ F IND (B[1 .. i − 1 ] , B [i])
8 then output B [i]
T (N) = ∑ T F IND (i)
i =1
T (N) = ∑ O (i) =
i =1
1 for i ← 1 to length (A)
2 do
3 if ¬ F IND (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ F IND (A , B [i]) ∧¬ F IND (B[1 .. i − 1 ] , B [i])
8 then output B [i]
T (N) = ∑ T F IND (i)
i =1
T (N) = ∑ O (i) = O(
N (N + 1)
i =1
1 for i ← 1 to length (A)
2 do
3 if ¬ F IND (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ F IND (A , B [i]) ∧¬ F IND (B[1 .. i − 1 ] , B [i])
8 then output B [i]
T (N) = ∑ T F IND (i)
i =1
T (N) = ∑ O (i) = O(
N (N + 1)
) = O(N 2 )
i =1 i =1
B INARY S EARCH (A , key )
1 first ← 1
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
N N N 2 4 ···
1 first ← 1 14
2 last ← length (A)
3 while first ≤ last
4 do middle ←⌈ (first + last) / 2 ⌉
5 if A [middle] = key
10 key
6 then return TRUE
7 elseif first = last
8 then return FALSE
9 elseif A [middle] > key
10 then last ← middle − 1 4
11 else fisrt ← middle +1
12 return FALSE
N N T N (N) = O(log N) 2 4 ···
Given two sorted sequences A = { a 1 , a 2 ,..., a n } and
B = { b 1 , b 2 ,..., b m } , where a 1 ≤ a 2 ≤ ... ≤ a n } and
b 1 ≤ b 2 ≤ ... ≤ b m , output a sequence X = { x 1 , x 2 ,..., x l } such that
◮ every element of A appears once in X ◮ every element of B appears once in X ◮ every element of X appears in A or in B or in both
1 for i ← 1 to length (A)
2 do
3 if ¬ B INARY S EARCH (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ B INARY S EARCH (A , B [i]) 8 ∧¬ B INARY S EARCH (B[1 .. i − 1 ] , B [i])
9 then output B [i]
1 for i ← 1 to length (A)
2 do
3 if ¬ B INARY S EARCH (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ B INARY S EARCH (A , B [i]) 8 ∧¬ B INARY S EARCH (B[1 .. i − 1 ] , B [i])
9 then output B [i]
T (N) = ∑ O (log i) =
i =1
1 for i ← 1 to length (A)
2 do
3 if ¬ B INARY S EARCH (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ B INARY S EARCH (A , B [i]) 8 ∧¬ B INARY S EARCH (B[1 .. i − 1 ] , B [i])
9 then output B [i]
T (N) = ∑ O (log i) = O(N log N)
i =1
1 for i ← 1 to length (A)
2 do
3 if ¬ B INARY S EARCH (A[1 .. i − 1 ] , A [i])
4 then output A [i]
5 for i ← 1 to length (B)
6 do
7 if ¬ B INARY S EARCH (A , B [i]) 8 ∧¬ B INARY S EARCH (B[1 .. i − 1 ] , B [i])
9 then output B [i]
T (N) = ∑ O (log i) = O(N log N)
i =1
better than O (N 2 ), but can we do even better than O(N log N)? better than O (N 2 ), but can we do even better than O(N log N)?
A = { 3 , 7 , 12 , 13 , 34 , 37 , 70 , 75 , 80 }
B = { 1 , 5 , 6 , 7 , 34 , 35 , 40 , 41 , 43 } B = { 1 , 5 , 6 , 7 , 34 , 35 , 40 , 41 , 43 }
A = { 3 , 7 , 12 , 13 , 34 , 37 , 70 , 75 , 80 }
B = { 1 , 5 , 6 , 7 , 34 , 35 , 40 , 41 , 43 }
so just like in B INARY S EARCH
I can avoid looking for an element of x if the first element I see is y > x I can avoid looking for an element of x if the first element I see is y > x
A = { 3 , 7 , 12 , 13 , 34 , 37 , 70 , 75 , 80 }
B = { 1 , 5 , 6 , 7 , 34 , 35 , 40 , 41 , 43 }
so just like in B INARY S EARCH
I can avoid looking for an element of x if the first element I see is y > x
High-level algorithm strategy ◮ step through every position i of A and every position j of B; at
each step ◮ output a i and advance i if a i ≤ b j or if j has reached the end of B, or ◮ output b j and advance j if a i ≥ b j or if i has reached the end of A
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 B 1 5 6 7 34 35 40 41 43
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =1
Output: Output:
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =1
Output: Output:
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =2
Output: 1 Output: 1
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =2
Output: 1 Output: 1
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =2
Output: 1 3 Output: 1 3
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =2
Output: 1 3 Output: 1 3
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =3
Output: 1 3 5 Output: 1 3 5
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =3
Output: 1 3 5 Output: 1 3 5
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =4
Output: 1 3 5 6 Output: 1 3 5 6
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =4
Output: 1 3 5 6 Output: 1 3 5 6
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =5
Output: 1 3 5 6 7 Output: 1 3 5 6 7
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =5
Output: 1 3 5 6 7 Output: 1 3 5 6 7
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =5
Output: 1 3 5 6 7 12 Output: 1 3 5 6 7 12
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =5
Output: 1 3 5 6 7 12 Output: 1 3 5 6 7 12
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =5
Output: 1 3 5 6 7 12 13 Output: 1 3 5 6 7 12 13
A 3 7 12 13 34 37 70 75 80
B 1 5 6 7 34 35 40 41 43 j =5
Output: 1 3 5 6 7 12 13. . .
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i > length (A)
5 then X ← X ◦ B [j] ✄ appends B[j] to X
6 j ← j +1
7 elseif j > length (B)
8 then X ← X ◦ A [i]
9 i ← i +1
10 elseif A [i] < B [j]
11 then X ← X ◦ A [i]
12 i ← i +1
13 else X ← X ◦ B [j]
14 j ← j +1 15 return X
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i > length (A)
5 then X ← X ◦ B [j] ✄ appends B[j] to X
6 j ← j +1
7 elseif j > length (B)
8 then X ← X ◦ A [i]
9 i ← i +1
10 elseif A [i] < B [j]
11 then X ← X ◦ A [i]
12 i ← i +1
13 else X ← X ◦ B [j]
14 j ← j +1 15 return X
◮ this algorithm is incorrect! (Exercise: fix it)
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i ≤ length (A) ∧ (j > length (B) ∨ A [i] < B [j])
5 then X ← X ◦ A [i]
6 i ← i +1
7 else X ← X ◦ B [j]
8 j ← j +1
9 return X
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i ≤ length (A) ∧ (j > length (B) ∨ A [i] < B [j])
5 then X ← X ◦ A [i]
6 i ← i +1
7 else X ← X ◦ B [j]
8 j ← j +1
9 return X
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i ≤ length (A) ∧ (j > length (B) ∨ A [i] < B [j])
5 then X ← X ◦ A [i]
6 i ← i +1
7 else X ← X ◦ B [j]
8 j ← j +1
9 return X T (N) = Θ(N)
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i ≤ length (A) ∧ (j > length (B) ∨ A [i] < B [j])
5 then X ← X ◦ A [i]
6 i ← i +1
7 else X ← X ◦ B [j]
8 j ← j +1
9 return X
T (N) = Θ(N)
Can we do better?
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i ≤ length (A) ∧ (j > length (B) ∨ A [i] < B [j])
5 then X ← X ◦ A [i]
6 i ← i +1
7 else X ← X ◦ B [j]
8 j ← j +1
9 return X
T (N) = Θ(N)
Can we do better? No!
2 X ←∅
3 while i ≤ length (A) ∨ j ≤ length (B)
4 do if i ≤ length (A) ∧ (j > length (B) ∨ A [i] < B [j])
5 then X ← X ◦ A [i]
6 i ← i +1
7 else X ← X ◦ B [j]
8 j ← j +1
9 return X
T (N) = Θ(N)
Can we do better? No! ◮ we have to output N = length(A) + length(B) elements
◮ merges two sorted sequences ◮ produces a sorted sequence
◮ merges two sorted sequences ◮ produces a sorted sequence
We could use that to implement a sort algorithm
◮ merges two sorted sequences ◮ produces a sorted sequence
We could use that to implement a sort algorithm Idea
◮ assume that two parts, A L and A R , of the input sequence A have
already been sorted
◮ merges two sorted sequences ◮ produces a sorted sequence
We could use that to implement a sort algorithm Idea
◮ assume that two parts, A L and A R , of the input sequence A have
already been sorted ◮ then you can use M ERGE to combine them to obtain the final
result
◮ merges two sorted sequences ◮ produces a sorted sequence
We could use that to implement a sort algorithm Idea
◮ assume that two parts, A L and A R , of the input sequence A have
already been sorted ◮ then you can use M ERGE to combine them to obtain the final
result ◮ this suggests a recursive algorithm
1 if length (A) = 1
2 then return A
3 m ←⌊ length (A) / 2 ⌋
4 A L ← M ERGE S ORT (A[1 ... m ])
5 A R ← M ERGE S ORT (A[m + 1 ... length (A)])
6 return M ERGE (A L , A R )
1 if length (A) = 1
2 then return A
3 m ←⌊ length (A) / 2 ⌋
4 A L ← M ERGE S ORT (A[1 ... m ])
5 A R ← M ERGE S ORT (A[m + 1 ... length (A)])
6 return M ERGE (A L , A R )
The complexity of M ERGE S ORT is
1 if length (A) = 1
2 then return A
3 m ←⌊ length (A) / 2 ⌋
4 A L ← M ERGE S ORT (A[1 ... m ])
5 A R ← M ERGE S ORT (A[m + 1 ... length (A)])
6 return M ERGE (A L , A R )
The complexity of M ERGE S ORT is
T (N) = 2T (N / 2 ) + O(N)
1 if length (A) = 1
2 then return A
3 m ←⌊ length (A) / 2 ⌋
4 A L ← M ERGE S ORT (A[1 ... m ])
5 A R ← M ERGE S ORT (A[m + 1 ... length (A)])
6 return M ERGE (A L , A R )
The complexity of M ERGE S ORT is
T (N) = 2T (N / 2 ) + O(N)
T (N) = O(N log N)
General strategy: given a problem P on input data A ◮ divide the input A into parts A 1 , A 2 ,..., A k with | A i |<| A | =N
◮ solve problem P for the individual k parts ◮ combine the partial solutions to obtain the solution for A
General strategy: given a problem P on input data A ◮ divide the input A into parts A 1 , A 2 ,..., A k with | A i |<| A | =N
◮ solve problem P for the individual k parts ◮ combine the partial solutions to obtain the solution for A
Complexity analysis
T (N) = T divide + ∑ T ( | A i | )+T combine
i =1
we will analyze this formula another time. . .
1 if length (A) = 0
2 then return B
3 if length (B) = 0
4 then return A
5 if A [1] < B [1]
6 then return A [1] ◦ M ERGE R (A[2 ... length (A)] , B )
7 else return B [1] ◦ M ERGE R (A , B [2 ... length (B)])
1 if length (A) = 0
2 then return B
3 if length (B) = 0
4 then return A
5 if A [1] < B [1]
6 then return A [1] ◦ M ERGE R (A[2 ... length (A)] , B )
7 else return B [1] ◦ M ERGE R (A , B [2 ... length (B)])
◮ again, this algorithm is a bit incorrect (Exercise: Fix it.)
1 if length (A) = 0
2 then return B
3 if length (B) = 0
4 then return A
5 if A [1] < B [1]
6 then return A [1] ◦ M ERGE R (A[2 ... length (A)] , B )
7 else return B [1] ◦ M ERGE R (A , B [2 ... length (B)])
◮ again, this algorithm is a bit incorrect (Exercise: Fix it.) The complexity of M ERGE R is
1 if length (A) = 0
2 then return B
3 if length (B) = 0
4 then return A
5 if A [1] < B [1]
6 then return A [1] ◦ M ERGE R (A[2 ... length (A)] , B )
7 else return B [1] ◦ M ERGE R (A , B [2 ... length (B)])
◮ again, this algorithm is a bit incorrect (Exercise: Fix it.) The complexity of M ERGE R is
T (N) = C 1 + T (N − 1 )
1 if length (A) = 0
2 then return B
3 if length (B) = 0
4 then return A
5 if A [1] < B [1]
6 then return A [1] ◦ M ERGE R (A[2 ... length (A)] , B )
7 else return B [1] ◦ M ERGE R (A , B [2 ... length (B)])
◮ again, this algorithm is a bit incorrect (Exercise: Fix it.) The complexity of M ERGE R is
T (N) = C 1 + T (N − 1 )=C 1 N
1 if length (A) = 0
2 then return B
3 if length (B) = 0
4 then return A
5 if A [1] < B [1]
6 then return A [1] ◦ M ERGE R (A[2 ... length (A)] , B )
7 else return B [1] ◦ M ERGE R (A , B [2 ... length (B)])
◮ again, this algorithm is a bit incorrect (Exercise: Fix it.) The complexity of M ERGE R is
T (N) = C 1 + T (N − 1 )=C 1 N = O(N)
Can we do better?
1 if length (A) = 0
2 then return B
3 if length (B) = 0
4 then return A
5 if A [1] < B [1]
6 then return A [1] ◦ M ERGE R (A[2 ... length (A)] , B )
7 else return B [1] ◦ M ERGE R (A , B [2 ... length (B)])
◮ again, this algorithm is a bit incorrect (Exercise: Fix it.) The complexity of M ERGE R is
T (N) = C 1 + T (N − 1 )=C 1 N = O(N)
Can we do better? No! (We knew that already)
and and
which means x =2 ℓ/ 2 x
=2 ℓ/ x 2 L y L +2 (x L y R +x R y L )+x R y R we reduced the problem of multiplying two numbers of ℓ bits into
the problem of multiplying four numbers of ℓ/ 2 bits. . .
and
which means x =2 ℓ/ 2 x L +x
ℓ x y ℓ/ =2 2 L L +2 (x L y R +x R y L )+x R y R we reduced the problem of multiplying two numbers of ℓ bits into
the problem of multiplying four numbers of ℓ/ 2 bits. . .
T ( ℓ ) = 4T ( ℓ/ 2 ) + O( ℓ ) T ( ℓ ) = 4T ( ℓ/ 2 ) + O( ℓ )
Y R which means x =2 ℓ/ 2 x L +x R and y =2 ℓ/ 2 y L +y R , so. . .
L +2 (x L y R +x R y L )+x R y R we reduced the problem of multiplying two numbers of ℓ bits into
=2 ℓ x
ℓ/ 2
the problem of multiplying four numbers of ℓ/ 2 bits. . .
T ( ℓ ) = 4T ( ℓ/ 2 ) + O( ℓ ) T ( ℓ ) = Θ( ℓ 2 )
xy
ℓ/ 2 x ℓ/ = (2 2
L +x R )(2 y L +y R )
=2 ℓ x
L y L +2 (x L y R +x R y L )+x R y R
L +x R )(2 y L +y R )
=2 ℓ x L y L +2 ℓ/ 2 (x L y R +x R y L )+x R y R but notice that x L y R +x R y L = (x L +x R )(y R +y L ) − x L y L − x R y R ,
so
=2 ℓ x y +2 ℓ/ 2 L L (x L y R +x R y L )+x R y R but notice that x L y R +x R y L = (x L +x R )(y R +y L ) − x L y L − x R y R ,
so xy =2 ℓ x L y L +2 ℓ/ 2 ((x L +x R )(y R +y L ) − x L y L − x R y R )+x R y R
=2 ℓ x L y L +2 ℓ/ 2 (x L y R +x R y L )+x R y R but notice that x L y R +x R y L = (x L +x R )(y R +y L ) − x L y L − x R y R ,
so xy =2 ℓ x y +2 ℓ/ 2 L L ((x L +x R )(y R +y L ) − x L y L − x R y R )+x R y R Only 3 multiplications: x L y L , (x L +x R )(y R +y L ), and x R y R
=2 ℓ x L y L +2 ℓ/ 2 (x L y R +x R y L )+x R y R but notice that x L y R +x R y L = (x L +x R )(y R +y L ) − x L y L − x R y R ,
so xy =2 ℓ x L y L +2 ℓ/ 2 ((x L +x R )(y R +y L ) − x L y L − x R y R )+x R y R Only 3 multiplications: x L y L , (x L +x R )(y R +y L ), and x R y R
T ( ℓ )= 3 T ( ℓ/ 2 ) + O( ℓ ) T ( ℓ )= 3 T ( ℓ/ 2 ) + O( ℓ )
L +x R )(2 y L +y R )
=2 ℓ x L y L +2 ℓ/ 2 (x L y R +x R y L )+x R y R but notice that x L y R +x R y L = (x L +x R )(y R +y L ) − x L y L − x R y R ,
so xy =2 ℓ x L y
L +2 ((x L +x R )(y R +y L ) − x L y L − x R y R )+x R y R Only 3 multiplications: x L y L , (x L +x R )(y R +y L ), and x R y R
T ( ℓ )= 3 T ( ℓ/ 2 ) + O( ℓ )
which, as we will see, leads to a much better complexity T
( . ℓ ) = O( ℓ log 2 3 ) = O( ℓ 1 59 ) ( . ℓ ) = O( ℓ log 2 3 ) = O( ℓ 1 59 )
Idea: first sort, then pick the element in the middle S IMPLE M EDIAN (A)
1 X ← M ERGE S ORT (A)
2 return X [ ⌊ length (A) / 2 ⌋ ] 2 return X [ ⌊ length (A) / 2 ⌋ ]
Idea: first sort, then pick the element in the middle S IMPLE M EDIAN (A)
1 X ← M ERGE S ORT (A)
2 return X [ ⌊ length (A) / 2 ⌋ ]
Is it correct? Is it correct?
Idea: first sort, then pick the element in the middle S IMPLE M EDIAN (A)
1 X ← M ERGE S ORT (A)
2 return X [ ⌊ length (A) / 2 ⌋ ]
Is it correct? Yes Is it correct? Yes
Idea: first sort, then pick the element in the middle S IMPLE M EDIAN (A)
1 X ← M ERGE S ORT (A)
2 return X [ ⌊ length (A) / 2 ⌋ ]
Is it correct? Yes How long does it take? Is it correct? Yes How long does it take?
Idea: first sort, then pick the element in the middle S IMPLE M EDIAN (A)
1 X ← M ERGE S ORT (A)
2 return X [ ⌊ length (A) / 2 ⌋ ]
Is it correct? Yes How long does it take? T (N) = O(N log N) Is it correct? Yes How long does it take? T (N) = O(N log N)
Idea: first sort, then pick the element in the middle S IMPLE M EDIAN (A)
1 X ← M ERGE S ORT (A)
2 return X [ ⌊ length (A) / 2 ⌋ ]
Is it correct? Yes How long does it take? T (N) = O(N log N) Can we do better? Is it correct? Yes How long does it take? T (N) = O(N log N) Can we do better?
Idea: first sort, then pick the element in the middle S IMPLE M EDIAN (A)
1 X ← M ERGE S ORT (A)
2 return X [ ⌊ length (A) / 2 ⌋ ]
Is it correct? Yes How long does it take? T (N) = O(N log N) Can we do better? Let’s try divide-and-conquer. . .
values in A are smaller than m and half are bigger than m values in A are smaller than m and half are bigger than m
value v ∈
A such that | A |− k elements of A are greater or equal to v A such that | A |− k elements of A are greater or equal to v
value v ∈
A such that | A |− k elements of A are greater or equal to v
E.g., ◮ for k = 1, the minimum of A E.g., ◮ for k = 1, the minimum of A
value v ∈
A such that | A |− k elements of A are greater or equal to v
E.g., ◮ for k = 1, the minimum of A
◮ for k = ⌊| A |/ 2 ⌋ , the median of A ◮ for k = ⌊| A |/ 2 ⌋ , the median of A
value v ∈
A such that | A |− k elements of A are greater or equal to v
E.g., ◮ for k = 1, the minimum of A
◮ for k = ⌊| A |/ 2 ⌋ , the median of A ◮ what is the 6th smallest element of
A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 } ? A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 } ?
value v ∈
A such that | A |− k elements of A are greater or equal to v
E.g., ◮ for k = 1, the minimum of A
◮ for k = ⌊| A |/ 2 ⌋ , the median of A ◮ what is the 6th smallest element of
A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 } ?
the 6th smallest element of A—a.k.a. select (A , 6 )—is 8
value v ∈ A
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v ◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
E.g., A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 }
and we must compute the 7th smallest value in A and we must compute the 7th smallest value in A
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
E.g., A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 }
and we must compute the 7th smallest value in A we pick a splitting value, say v =5 and we must compute the 7th smallest value in A we pick a splitting value, say v =5
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
E.g., A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 }
and we must compute the 7th smallest value in A we pick a splitting value, say v =5 and we must compute the 7th smallest value in A we pick a splitting value, say v =5
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
E.g., A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 }
and we must compute the 7th smallest value in A we pick a splitting value, say v =5 and we must compute the 7th smallest value in A we pick a splitting value, say v =5
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
E.g., A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 }
and we must compute the 7th smallest value in A we pick a splitting value, say v =5
A L = { 2 , 4 , 1 } A v = { 5 , 5 } A R = { 36 , 21 , 8 , 13 , 11 , 20 } A L = { 2 , 4 , 1 } A v = { 5 , 5 } A R = { 36 , 21 , 8 , 13 , 11 , 20 }
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
E.g., A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 }
and we must compute the 7th smallest value in A we pick a splitting value, say v =5
A L = { 2 , 4 , 1 } A v = { 5 , 5 } A R = { 36 , 21 , 8 , 13 , 11 , 20 } Now, where is the 7th smallest value of A? A L = { 2 , 4 , 1 } A v = { 5 , 5 } A R = { 36 , 21 , 8 , 13 , 11 , 20 } Now, where is the 7th smallest value of A?
◮ A L contains the set of elements that are less than v ◮ A v contains the set of elements that are equal to v ◮ A R contains the set of elements that are greater then v
E.g., A = { 2 , 36 , 5 , 21 , 8 , 13 , 11 , 20 , 5 , 4 , 1 }
and we must compute the 7th smallest value in A we pick a splitting value, say v =5
A L = { 2 , 4 , 1 } A v = { 5 , 5 } A R = { 36 , 21 , 8 , 13 , 11 , 20 } Now, where is the 7th smallest value of A?
It is the 2nd smallest value of A R
select (A L , k ) if k ≤| A L | select (A , k )= v
if | A L |< k ≤| A L | + | A v | select (A
R , k −| A L |−| A v | ) if k >| A L | + | A v |
select (A L , k ) if k ≤| A L | select (A , k )= v
if | A L |< k ≤| A L | + | A v | select (A
R , k −| A L |−| A v | ) if k >| A L | + | A v |
Computing A L ,A v , and A R takes O (N) steps
select (A L , k ) if k ≤| A L | select (A , k )= v
if | A L |< k ≤| A L | + | A v | select (A
R , k −| A L |−| A v | ) if k >| A L | + | A v |
Computing A L ,A v , and A R takes O (N) steps How do we pick v?
select (A L , k ) if k ≤| A L | select (A , k )= v
if | A L |< k ≤| A L | + | A v | select (A
R , k −| A L |−| A v | ) if k >| A L | + | A v |
Computing A L ,A v , and A R takes O (N) steps How do we pick v?
Ideally, we should pick v so as to obtain | A L |≈| A R |≈| A |/ 2
◮ so, ideally we should pick v = median(A), but. . .
select (A L , k ) if k ≤| A L | select (A , k )= v
if | A L |< k ≤| A L | + | A v | select (A
R , k −| A L |−| A v | ) if k >| A L | + | A v |
Computing A L ,A v , and A R takes O (N) steps How do we pick v?
Ideally, we should pick v so as to obtain | A L |≈| A R |≈| A |/ 2
◮ so, ideally we should pick v = median(A), but. . . We pick a random element of A
S ELECTION (A , k )
1 v ← A [random(1 ... | A | )]
2 A L , A v , A R ←∅
3 for i ← 1 to | A |
4 do if A [i] < v
5 then A L ← A L ∪ A [i]
6 elseif A [i] = v
7 then A v ← A v ∪ A [i]
8 else A R ← A R ∪ A [i]
9 if k ≤| A L |
10 then return S ELECTION (A L , k )
11 elseif k >| A L | + | A v | return S ELECTION (A R , k −| A L |−| A v | )
12 else return v