P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Let’s create a 2m × n data measurement matrix:
37
B. Leibe
mn m
m n
n
x x
x x
x x
x x
x
D
ˆ ˆ
ˆ ˆ
ˆ ˆ
ˆ ˆ
ˆ
2 1
2 22
21 1
12 11
Cameras 2
m
Points n
C. Tomasi and T. Kanade. Shape and motion from image streams under orthography:
A factorization method. IJCV, 92:137-154, November 1992.
Slide credit: Svetlana Lazebnik
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Let’s create a 2m × n data measurement matrix:
•
The measurement matrix
D = MS
must have rank 3
38
B. Leibe
C. Tomasi and T. Kanade. Shape and motion from image streams under orthography:
A factorization method. IJCV, 92:137-154, November 1992.
Slide credit: Svetlana Lazebnik
Cameras 2
m × 3
n
m mn
m m
n n
X X
X
A A
A
x x
x x
x x
x x
x
D
2 1
2 1
2 1
2 22
21 1
12 11
ˆ ˆ
ˆ ˆ
ˆ ˆ
ˆ ˆ
ˆ
Points 3 × n
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
39
B. Leibe
Slide credit: Martial Hebert
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Singular value decomposition of D:
40 Slide credit: Martial Hebert
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Singular value decomposition of D:
41 Slide credit: Martial Hebert
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Obtaining a factorization from SVD:
42 Slide credit: Martial Hebert
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Obtaining a factorization from SVD:
43 Slide credit: Martial Hebert
This decomposition minimizes
|D-MS|
2
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
The decomposition is not unique. We get the same D by using any 3×3 matrix C and applying the
transformations M → MC, S →C
-1
S.
•
That is because we have only an affine transformation and we have not enforced any Euclidean constraints
like forcing the image axis to be perpendicular, for example. We need a Euclidean upgrade.
44
B. Leibe
Slide credit: Martial Hebert
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Orthographic assumption: image axes are perpendicular and scale is 1.
•
This can be converted into a system of 3m equations:
45
B. Leibe
x
X a
1
a
2
a
1
· a
2
= 0
|a
1
|
2
= |a
2
|
2
= 1
Slide adapted from S. Lazebnik, M. Hebert
1 2
1 2
1 1
1 2
2 2
ˆ ˆ
ˆ 1
1 , 1,...,
ˆ 1
1
T T
i i
i i
T T
i i
i T
T i
i i
a a a CC a
a a CC a
i m
a a CC a
�
� �
� �
�
� �
� �
� �
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
This can be converted into a system of 3m equations:
•
Let
•
Then this translates to 3m equations in L
Solve for L
Recover C from L by Cholesky decomposition: L = CC
T
Update M and S: M = MC, S = C
-1
S
46
B. Leibe
Slide adapted from S. Lazebnik, M. Hebert
1 2
1 2
1 1
1 2
2 2
ˆ ˆ
ˆ 1
1 , 1,...,
ˆ 1
1
T T
i i
i i
T T
i i
i T
T i
i i
a a a CC a
a a CC a
i m
a a CC a
�
� �
� �
�
� �
� �
� �
1 2
, 1,...,
T i
i T
i
a A
i m
a � �
� � � �
, 1,...,
T i
i
A LA I
i m
T
L CC
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
•
Given: m images and n features x
ij
•
For each image i, center the feature coordinates.
•
Construct a 2m × n measurement matrix D:
Column j contains the projection of point j in all views
Row i contains one coordinate of the projections of all the n points in image i
•
Factorize D:
Compute SVD: D = U W V
T
Create U
3
by taking the first 3 columns of U
Create V
3
by taking the first 3 columns of V
Create W
3
by taking the upper left 3 × 3 block of W
•
Create the motion and shape matrices:
M = U
3
W
3½
and S = W
3½
V
3T
or M = U
3
and S = W
3
V
3T
•
Eliminate affine ambiguity
47 Slide credit: Martial Hebert
P e
rc e
p tu
a l
a n
d S
e n
s o
ry A
u g
m e
n te
d C
o m
p u
ti n
g
C o
m p
u te
r V
is io
n W
S 8
9
48
B. Leibe
C. Tomasi and T. Kanade. Shape and motion from image streams under orthography:
