warping.ppt 1959KB Jun 23 2011 12:10:26 PM
Image Warping
15-463: Computational Photography Alexei Efros, CMU, Fall 2005
Some slides from Steve Seitz
(2)
Image Warping
image filtering: change range of image
g(x) = T(f(x))
f
x
T
f
x
f
x
T
f
x
image warping: change domain of image
(3)
Image Warping
T
T
f
f g
g
image filtering: change range of image
g(x) = h(T(x))
image warping: change domain of image
(4)
Parametric (global) warping
Examples of parametric warps:
translation rotation aspect
(5)
Parametric (global) warping
Transformation T is a coordinate-changing machine: p’ = T(p)
What does it mean that T is global?
• Is the same for any point p
• can be described by just a few numbers (parameters)
Let’s represent T as a matrix: p’ = M*p
T
p = (x,y) p’ = (x’,y’)
y
x
y
x
M
'
'
(6)
Scaling
Scaling a coordinate means multiplying each of its components by a scalar
Uniform scaling means this scalar is the same for all components:
(7)
Non-uniform scaling: different scalars per component:
Scaling
X 2,
(8)
Scaling
Scaling operation:
Or, in matrix form:
by
y
ax
x
'
'
y
x
b
a
y
x
0
0
'
'
scaling matrix S
(9)
2-D Rotation
(x, y)
(x’, y’)
x’ = x
cos
(
) - y
sin
(
)
y’ = x
sin
(
) + y
cos
(
)
(10)
2-D Rotation
x = r cos ()
y = r sin ()
x’ = r cos ( + )
y’ = r sin ( + )
Trig Identity…
x’ = r cos() cos() – r sin() sin()
y’ = r sin() sin() + r cos() cos()
Substitute…
x’ = x cos() - y sin()
y’ = x sin() + y cos()
(x, y)
(x’, y’)
(11)
2-D Rotation
This is easy to capture in matrix form:
Even though sin() and cos() are nonlinear functions of ,
• x’ is a linear combination of x and y • y’ is a linear combination of x and y
What is the inverse transformation?
• Rotation by –
• For rotation matrices, det(R) = 1 so
y
x
y
x
cos
sin
sin
cos
'
'
T
R R1
(12)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Identity?
y
y
x
x
'
'
y
x
y
x
1
0
0
1
'
'
2D Scale around (0,0)?
y
s
y
x
s
x
y x*
'
*
'
y
x
s
s
y
x
y x0
0
'
'
(13)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Rotate around (0,0)?
y
x
y
x
y
x
*
cos
*
sin
'
cos
*
sin
*
'
y x y x cos sin sin cos ' '2D Shear?
y
x
sh
y
y
sh
x
x
y x
*
'
*
'
y
x
sh
sh
y
x
y x1
1
'
'
(14)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Mirror about Y axis?
y
y
x
x
'
'
y
x
y
x
1
0
1
0
'
'
2D Mirror over (0,0)?
y
y
x
x
'
'
y
x
y
x
1
0
1
0
'
(15)
2x2 Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Translation?
y x
t
y
y
t
x
x
'
'
Only linear 2D transformations
can be represented with a 2x2 matrix
NO!(16)
All 2D Linear Transformations
Linear transformations are combinations of …
• Scale, • Rotation, • Shear, and • Mirror
Properties of linear transformations:
• Origin maps to origin • Lines map to lines
• Parallel lines remain parallel • Ratios are preserved
• Closed under composition
y x d c b a y x ' '
y
x
l
k
j
i
h
g
f
e
d
c
b
a
y
x
'
'
(17)
Linear Transformations as Change of Basis
Any linear transformation is a basis!!!
• What’s the inverse transform?
• How can we change from any basis to any basis? • What if the basis are orthogonal?
j =(0,1)
i =(1,0)
p
p=4i+3j = (4,3) p’=4u+3v px’=4ux+3vx
py’=4uy+3vy
v =(vx,vy)
u=(ux,uy)
p’
p
p
y y
x x
y y
x x
v
u
v
u
v
u
v
u
3
4
'
(18)
Homogeneous Coordinates
Q: How can we represent translation as a 3x3 matrix?
y x
t
y
y
t
x
x
'
'
(19)
Homogeneous Coordinates
Homogeneous coordinates
• represent coordinates in 2 dimensions with a 3-vector
1
coords s
homogeneou
y
x
y
(20)
Homogeneous Coordinates
Q: How can we represent translation as a 3x3 matrix?
A: Using the rightmost column:
1
0
0
1
0
0
1
y x
t
t
ranslation
T
y x
t
y
y
t
x
x
'
'
(21)
Translation
Example of translation
1 1 1 0 0 1 0 0 1 1 ' ' y x y x t y t x y x t t y x
tx = 2 ty = 1
(22)
Homogeneous Coordinates
Add a 3rd coordinate to every 2D point
• (x, y, w) represents a point at location (x/w, y/w) • (x, y, 0) represents a point at infinity
• (0, 0, 0) is not allowed
Convenient
coordinate system to represent many
useful
transformations
1 2 1
2
(2,1,1) or (4,2,2) or (6,3,3)
x y
(23)
Basic 2D Transformations
Basic 2D transformations as 3x3 matrices
1 1 0 0 0 cos sin 0 sin cos 1 ' ' y x y x 1 1 0 0 1 0 0 1 1 ' ' y x t t y x y x 1 1 0 0 0 1 0 1 1 ' ' y x sh sh y x y x Translate Rotate Shear 1 1 0 0 0 0 0 0 1 ' ' y x s s y x y x Scale
(24)
Affine Transformations
Affine transformations are combinations of …
• Linear transformations, and • Translations
Properties of affine transformations:
• Origin does not necessarily map to origin
• Lines map to lines
• Parallel lines remain parallel • Ratios are preserved
• Closed under composition • Models change of basis
w y x f
e
da b c w
y x
1 0
0 '
(25)
Projective Transformations
Projective transformations …
• Affine transformations, and • Projective warps
Properties of projective transformations: • Origin does not necessarily map to origin
• Lines map to lines
• Parallel lines do not necessarily remain parallel • Ratios are not preserved
• Closed under composition • Models change of basis
w y x i
h g
f e
da b c w
y x
' ' '
(26)
Matrix Composition
Transformations can be combined by matrix multiplication
w
y
x
sy
sx
ty
tx
w
y
x
1
0
0
0
0
0
0
1
0
0
cos
0
sin
sin
0
cos
1
0
0
1
0
0
1
'
'
'
(27)
2D image transformations
These transformations are a nested set of groups
(28)
Image warping
Given a coordinate transform (x’,y’) = h(x,y) and a source image f(x,y), how do we compute a
transformed image g(x’,y’) = f(T(x,y))?
x x’
T(x,y)
f(x,y) g(x’,y’)
(29)
f(x,y) g(x’,y’)
Forward warping
Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: what if pixel lands “between” two pixels?
(30)
f(x,y) g(x’,y’)
Forward warping
Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: what if pixel lands “between” two pixels?
y y’
A: distribute color among neighboring pixels (x’,y’)
(31)
f(x,y) g(x’,y’)
x y
Inverse warping
Get each pixel g(x’,y’) from its corresponding location (x,y) = T-1(x’,y’) in the first image
x x’
Q: what if pixel comes from “between” two pixels?
y’ T-1(x,y)
(32)
f(x,y) g(x’,y’)
x y
Inverse warping
Get each pixel g(x’,y’) from its corresponding location (x,y) = T-1(x’,y’) in the first image
x x’
T-1(x,y)
Q: what if pixel comes from “between” two pixels?
y’
A: Interpolate color value from neighbors
(33)
Bilinear interpolation
(34)
Forward vs. inverse warping
Q: which is better?
A: usually inverse—eliminates holes
(1)
f(x,y) g(x’,y’)
Forward warping
Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: what if pixel lands “between” two pixels?
(2)
f(x,y) g(x’,y’)
Forward warping
Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: what if pixel lands “between” two pixels?
y y’
A: distribute color among neighboring pixels (x’,y’)
(3)
f(x,y) g(x’,y’)
x y
Inverse warping
Get each pixel g(x’,y’) from its corresponding location (x,y) = T-1(x’,y’) in the first image
x x’
Q: what if pixel comes from “between” two pixels?
y’ T-1(x,y)
(4)
f(x,y) g(x’,y’)
x y
Inverse warping
Get each pixel g(x’,y’) from its corresponding location (x,y) = T-1(x’,y’) in the first image
x x’
T-1(x,y)
Q: what if pixel comes from “between” two pixels?
y’
A: Interpolate color value from neighbors
(5)
Bilinear interpolation
(6)
Forward vs. inverse warping
Q: which is better?
A: usually inverse—eliminates holes