Lecture 4 Image Restoration
Lecture 4
Image Restoration
Image Restoration
• Image restoration: recover an image that has been
degraded by using a prior knowledge of the degradation
phenomenon.
• Model the degradation and applying the inverse process in
order to recover the original image.
2
A Model of Image Degradation/Restoration
Process
• Degradation
– Degradation function H
– Additive noise ( x, y )
3
A Model of Image Degradation/Restoration
Process
If H is a linear,
position-invariant process, then
the degraded image is given in the spatial domain by
g ( x, y ) h ( x , y ) f ( x , y ) ( x , y )
4
A Model of Image Degradation/Restoration
Process
The model of the degraded image is given in
the frequency domain by
G (u, v) H (u, v) F (u, v) N (u, v)
5
Noise Sources
• The principal sources of noise in digital images arise during
image acquisition and/or transmission
Image acquisition
e.g., light levels, sensor temperature, etc.
Transmission
e.g., lightning or other atmospheric disturbance in wireless
network
6
Noise Models (1)
• White noise
– The Fourier spectrum of noise is constant
• With the exception of spatially periodic noise, we assume
– Noise is independent of spatial coordinates
– Noise is uncorrelated with respect to the image itself
7
Noise Models (2)
Gaussian noise
Electronic circuit noise, sensor noise due to poor illumination and/or
high temperature
Rayleigh noise
Range imaging
8
Range Imaging (1)
Short for High Dynamic Range Imaging. HDRI is an imaging
technique that allows for a greater dynamic range of exposure
than would be obtained through any normal imaging process.
It is now popularly used to refer to the process of tone
mapping* together with bracketed** exposures of normal
digital images, giving the end result a high, often exaggerated
dynamic range
* Tone mapping is a technique used in image processing and computer graphics to map
a set of colours to another; often to approximate the appearance of HDRI in media with a
more limited dynamic range
** bracketing is the general technique of taking several shots of the same subject using
different or the same camera settings
9
Range Imaging (2)
The intention of HDRI is to accurately represent the wide
range of intensity levels found in real scenes ranging from
direct sunlight to shadows
HDRI, also called HDR (High Dynamic Range) is a feature
commonly found in high-end graphics and imaging software
10
Range Imaging: Examples (1)
Tower Bridge in
Sacramento,
CA
11
Range Imaging: Examples (2)
http://en.wikipedia.org/wiki/High_dynamic_range_imaging
Sydney Harbour
Bridge HDRi
produces greater
detail and fewer
shadows
12
Old Saint Paul’s
Wellinton, New Zealand
13
Noise Models (3)
Erlang (gamma) noise: Laser imaging
Exponential noise: Laser imaging
Uniform noise: Least descriptive; Basis for numerous random
number generators
Impulse noise: quick transients,
such as faulty switching
14
Gaussian Noise (1)
The PDF of Gaussian random variable, z, is given by
1
( z z )2 /2 2
p(z )
e
2
where, z represents intensity
z is the mean (average) value of z
is the standard deviation
15
Gaussian Noise (2)
The PDF of Gaussian random variable, z, is given by
1
( z z )2 /2 2
p(z )
e
2
– 70% of its values will be in the range
( ), ( )
– 95% of its values will be in the range
( 2 ), ( 2 )
16
Rayleigh Noise
The PDF of Rayleigh noise is given by
2
( z a )2 / b
for z a
( z a )e
p( z ) b
0
for z a
The mean and variance of this density are given by
z a b / 4
b(4 )
4
2
17
Erlang (Gamma) Noise
The PDF of Erlang noise is given by
a b z b 1 az
e
p ( z ) (b 1)!
0
for z 0
for z a
The mean and variance of this density are given by
z b/a
2 b / a2
18
Exponential Noise
The PDF of exponential noise is given by
ae az
p( z )
0
for z 0
for z a
The mean and variance of this density are given by
z 1/ a
2 1/ a 2
19
Uniform Noise
The PDF of uniform noise is given by
1
p( z ) b a
0
for a z b
otherwise
The mean and variance of this density are given by
z (a b ) / 2
2 (b a)2 /12
20
Impulse (Salt-and-Pepper) Noise
The PDF of (bipolar) impulse noise is given by
Pa
p ( z ) Pb
0
for z a
for z b
otherwise
if b a, gray-level b will appear as a light dot,
while level a will appear like a dark dot.
If either Pa or Pb is zero, the impulse noise is called
unipolar
21
22
Examples of Noise: Original Image
23
Examples of Noise: Noisy Images(1)
24
Examples of Noise: Noisy Images(2)
25
Periodic Noise
• Periodic noise in an image arises typically from electrical or
electromechanical interference during image acquisition.
• It is a type of spatially dependent noise
• Periodic noise can be reduced significantly via frequency
domain filtering
26
An Example of Periodic Noise
27
Estimation of Noise Parameters (1)
The shape of the histogram identifies the closest PDF match
28
Estimation of Noise Parameters (2)
Consider a subimage denoted by S , and let ps ( zi ), i 0, 1, ..., L -1,
denote the probability estimates of the intensities of the pixels in S .
The mean and variance of the pixels in S:
L 1
z zi ps ( zi )
i 0
L 1
and
2 ( zi z) 2 ps ( zi )
i 0
29
Restoration in the Presence of Noise Only
̶ Spatial Filtering
Noise model without degradation
g ( x , y ) f ( x , y ) ( x, y )
and
G (u , v) F (u, v) N (u , v)
30
Spatial Filtering: Mean Filters (1)
Let S xy represent the set of coordinates in a rectangle
subimage window of size m n, centered at ( x, y ).
Arithmetic mean filter
ˆf ( x, y ) 1
g ( s, t )
m.n ( s ,t )S
xy
31
Spatial Filtering: Mean Filters (2)
Geometric mean filter:
1
mn
fˆ ( x, y )
g ( s, t )
( s ,t )S
xy
Generally, a geometric mean filter achieves smoothing
comparable to the arithmetic mean filter, but it tends to
lose less image detail in the process
32
Spatial Filtering: Example
33
Spatial Filtering: Order-Statistic Filters (1)
Median filter:
fˆ ( x, y) mediang ( s, t )
( s ,t )S xy
Max filter:
fˆ ( x, y) max g ( s, t )
( s ,t )S xy
Min filter:
fˆ ( x, y) min g ( s, t )
( s ,t )S xy
34
Spatial Filtering: Order-Statistic Filters (2)
Mid-point filter:
ˆf ( x, y) 1 max g ( s, t ) min g ( s, t )
( s ,t )S
s
(
t
,
)
S
2
xy
xy
35
Spatial Filtering: Order-Statistic Filters (3)
Alpha-trim mean filter:
fˆ ( x, y )
1
mn d
g ( s, t )
r
( s ,t )S xy
We delete the d / 2 lowest and the d / 2 highest intensity values of
g ( s, t ) in the neighborhood S xy . Let g r ( s, t ) represent the remaining
mn - d pixels.
36
37
38
Spatial Filtering: Adaptive Filters
Adaptive filters
The behavior changes based on statistical characteristics
of the image inside the filter region defined by the mхn
rectangular window.
The performance is superior to that of the filters
discussed
39
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (1)
S xy : local region
The response of the filter at the center point (x,y) of S xy
is based on four quantities:
(a) g ( x, y ), the value of the noisy image at (x, y );
(b) 2 , the variance of the noise corrupting f ( x, y )
to form g ( x, y );
(c) mL , the local mean of the pixels in S xy ;
(d) L2 , the local variance of the pixels in S xy .
40
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (2)
The behavior of the filter:
(a) if 2 is zero, the filter should return simply the value
of g ( x, y ).
(b) if the local variance is high relative to 2 , the filter
should return a value close to g ( x, y );
(c) if the two variances are equal, the filter returns the
arithmetic mean value of the pixels in S xy .
41
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (3)
An adaptive expression for obtaining a filtered image
based on the assumptions:
fˆ ( x, y ) g ( x, y )
2
g
(
x
,
y
)
m
L
2
L
42
43
Adaptive Filters:
Adaptive Median Filters (1)
The notation:
zmin minimum intensity value in S xy
zmax maximum intensity value in S xy
zmed median intensity value in S xy
z xy intensity value at coordinates ( x, y )
S max maximum allowed size of S xy
44
Adaptive Filters:
Adaptive Median Filters (2)
The adaptive median-filtering works in two stages:
Stage A:
A1 = zmed zmin ;
A2 = zmed zmax
if A1>0 and A20 and B20 and A20 and B2 amplify noise !!!
64
Inverse Filtering
1. We can't exactly recover the undegraded image
because N (u, v) is not known.
2. If the degradation function has zero or very
small values, then the ratio N (u , v) / H (u, v) could
easily dominate the estimate F (u, v).
65
Inverse Filtering
EXAMPLE
The image in Fig. 5.25(b) was inverse filtered using the
exact inverse of the degradation function that generated
that image. That is, the degradation function is
H (u , v ) e
2
k u M /2 ( v N /2)2
5/6
, k 0.0025
66
The poor performance
of direct inverse
filtering in general
A Butterworth
lowpass
function of
order 10
67
Minimum Mean Square Error (Wiener)
Filtering
N. Wiener (1942)
Objective
Find an estimate of the uncorrupted image such that the mean
square error between them is minimized
2
ˆ
e E (f f)
2
68
Minimum Mean Square Error (Wiener)
Filtering
2
1
H
(
u
,
v
)
G (u, v)
Fˆ (u, v)
2
H (u, v) H (u, v) S (u, v) / S f (u, v)
H(u,v): degradation function
H*(u,v): complex conjugate of H(u,v)
|H(u,v)|2 = H(u,v)H*(u,v)
Sn(u,v)= |N(u,v)|2: power spectrum of the noise
Sf(u,v)= |F(u,v)|2 : power spectrum of the undegraded
image
69
Minimum Mean Square Error (Wiener)
Filtering
2
1
H
u
v
(
,
)
G(u, v)
Fˆ (u, v)
2
H (u, v) H (u, v) K
K is a specified constant. Generally, the value of K is
chosen interactively to yield the best visual results.
70
Minimum Mean Square Error (Wiener)
Filtering
71
Left:
degradated
image
Middle:
inverse
filtering
Right:
Wiener
filtering
72
Constrained Least Squares Filtering
• In Wiener filter, the power spectra of the undegraded
image and noise must be known. Although a constant
estimate is sometimes useful, it is not always suitable.
• Constrained least squares filtering just requires the mean
and variance of the noise.
73
Constrained Least Squares Filtering
H *(u, v)
F (u , v)
G (u, v)
2
2
| H (u, v) | | P(u, v) |
P(u , v) is the Fourier transform of the function
0 1 0
p ( x, y ) 1 4 1
0 1 0
is a parameter
(Regualarization term: ensure « smoothness » at high frequenciesTikhonov regularization - minimize the output of the Laplacian 74
operator ! )
Examples
75
Image Restoration
Image Restoration
• Image restoration: recover an image that has been
degraded by using a prior knowledge of the degradation
phenomenon.
• Model the degradation and applying the inverse process in
order to recover the original image.
2
A Model of Image Degradation/Restoration
Process
• Degradation
– Degradation function H
– Additive noise ( x, y )
3
A Model of Image Degradation/Restoration
Process
If H is a linear,
position-invariant process, then
the degraded image is given in the spatial domain by
g ( x, y ) h ( x , y ) f ( x , y ) ( x , y )
4
A Model of Image Degradation/Restoration
Process
The model of the degraded image is given in
the frequency domain by
G (u, v) H (u, v) F (u, v) N (u, v)
5
Noise Sources
• The principal sources of noise in digital images arise during
image acquisition and/or transmission
Image acquisition
e.g., light levels, sensor temperature, etc.
Transmission
e.g., lightning or other atmospheric disturbance in wireless
network
6
Noise Models (1)
• White noise
– The Fourier spectrum of noise is constant
• With the exception of spatially periodic noise, we assume
– Noise is independent of spatial coordinates
– Noise is uncorrelated with respect to the image itself
7
Noise Models (2)
Gaussian noise
Electronic circuit noise, sensor noise due to poor illumination and/or
high temperature
Rayleigh noise
Range imaging
8
Range Imaging (1)
Short for High Dynamic Range Imaging. HDRI is an imaging
technique that allows for a greater dynamic range of exposure
than would be obtained through any normal imaging process.
It is now popularly used to refer to the process of tone
mapping* together with bracketed** exposures of normal
digital images, giving the end result a high, often exaggerated
dynamic range
* Tone mapping is a technique used in image processing and computer graphics to map
a set of colours to another; often to approximate the appearance of HDRI in media with a
more limited dynamic range
** bracketing is the general technique of taking several shots of the same subject using
different or the same camera settings
9
Range Imaging (2)
The intention of HDRI is to accurately represent the wide
range of intensity levels found in real scenes ranging from
direct sunlight to shadows
HDRI, also called HDR (High Dynamic Range) is a feature
commonly found in high-end graphics and imaging software
10
Range Imaging: Examples (1)
Tower Bridge in
Sacramento,
CA
11
Range Imaging: Examples (2)
http://en.wikipedia.org/wiki/High_dynamic_range_imaging
Sydney Harbour
Bridge HDRi
produces greater
detail and fewer
shadows
12
Old Saint Paul’s
Wellinton, New Zealand
13
Noise Models (3)
Erlang (gamma) noise: Laser imaging
Exponential noise: Laser imaging
Uniform noise: Least descriptive; Basis for numerous random
number generators
Impulse noise: quick transients,
such as faulty switching
14
Gaussian Noise (1)
The PDF of Gaussian random variable, z, is given by
1
( z z )2 /2 2
p(z )
e
2
where, z represents intensity
z is the mean (average) value of z
is the standard deviation
15
Gaussian Noise (2)
The PDF of Gaussian random variable, z, is given by
1
( z z )2 /2 2
p(z )
e
2
– 70% of its values will be in the range
( ), ( )
– 95% of its values will be in the range
( 2 ), ( 2 )
16
Rayleigh Noise
The PDF of Rayleigh noise is given by
2
( z a )2 / b
for z a
( z a )e
p( z ) b
0
for z a
The mean and variance of this density are given by
z a b / 4
b(4 )
4
2
17
Erlang (Gamma) Noise
The PDF of Erlang noise is given by
a b z b 1 az
e
p ( z ) (b 1)!
0
for z 0
for z a
The mean and variance of this density are given by
z b/a
2 b / a2
18
Exponential Noise
The PDF of exponential noise is given by
ae az
p( z )
0
for z 0
for z a
The mean and variance of this density are given by
z 1/ a
2 1/ a 2
19
Uniform Noise
The PDF of uniform noise is given by
1
p( z ) b a
0
for a z b
otherwise
The mean and variance of this density are given by
z (a b ) / 2
2 (b a)2 /12
20
Impulse (Salt-and-Pepper) Noise
The PDF of (bipolar) impulse noise is given by
Pa
p ( z ) Pb
0
for z a
for z b
otherwise
if b a, gray-level b will appear as a light dot,
while level a will appear like a dark dot.
If either Pa or Pb is zero, the impulse noise is called
unipolar
21
22
Examples of Noise: Original Image
23
Examples of Noise: Noisy Images(1)
24
Examples of Noise: Noisy Images(2)
25
Periodic Noise
• Periodic noise in an image arises typically from electrical or
electromechanical interference during image acquisition.
• It is a type of spatially dependent noise
• Periodic noise can be reduced significantly via frequency
domain filtering
26
An Example of Periodic Noise
27
Estimation of Noise Parameters (1)
The shape of the histogram identifies the closest PDF match
28
Estimation of Noise Parameters (2)
Consider a subimage denoted by S , and let ps ( zi ), i 0, 1, ..., L -1,
denote the probability estimates of the intensities of the pixels in S .
The mean and variance of the pixels in S:
L 1
z zi ps ( zi )
i 0
L 1
and
2 ( zi z) 2 ps ( zi )
i 0
29
Restoration in the Presence of Noise Only
̶ Spatial Filtering
Noise model without degradation
g ( x , y ) f ( x , y ) ( x, y )
and
G (u , v) F (u, v) N (u , v)
30
Spatial Filtering: Mean Filters (1)
Let S xy represent the set of coordinates in a rectangle
subimage window of size m n, centered at ( x, y ).
Arithmetic mean filter
ˆf ( x, y ) 1
g ( s, t )
m.n ( s ,t )S
xy
31
Spatial Filtering: Mean Filters (2)
Geometric mean filter:
1
mn
fˆ ( x, y )
g ( s, t )
( s ,t )S
xy
Generally, a geometric mean filter achieves smoothing
comparable to the arithmetic mean filter, but it tends to
lose less image detail in the process
32
Spatial Filtering: Example
33
Spatial Filtering: Order-Statistic Filters (1)
Median filter:
fˆ ( x, y) mediang ( s, t )
( s ,t )S xy
Max filter:
fˆ ( x, y) max g ( s, t )
( s ,t )S xy
Min filter:
fˆ ( x, y) min g ( s, t )
( s ,t )S xy
34
Spatial Filtering: Order-Statistic Filters (2)
Mid-point filter:
ˆf ( x, y) 1 max g ( s, t ) min g ( s, t )
( s ,t )S
s
(
t
,
)
S
2
xy
xy
35
Spatial Filtering: Order-Statistic Filters (3)
Alpha-trim mean filter:
fˆ ( x, y )
1
mn d
g ( s, t )
r
( s ,t )S xy
We delete the d / 2 lowest and the d / 2 highest intensity values of
g ( s, t ) in the neighborhood S xy . Let g r ( s, t ) represent the remaining
mn - d pixels.
36
37
38
Spatial Filtering: Adaptive Filters
Adaptive filters
The behavior changes based on statistical characteristics
of the image inside the filter region defined by the mхn
rectangular window.
The performance is superior to that of the filters
discussed
39
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (1)
S xy : local region
The response of the filter at the center point (x,y) of S xy
is based on four quantities:
(a) g ( x, y ), the value of the noisy image at (x, y );
(b) 2 , the variance of the noise corrupting f ( x, y )
to form g ( x, y );
(c) mL , the local mean of the pixels in S xy ;
(d) L2 , the local variance of the pixels in S xy .
40
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (2)
The behavior of the filter:
(a) if 2 is zero, the filter should return simply the value
of g ( x, y ).
(b) if the local variance is high relative to 2 , the filter
should return a value close to g ( x, y );
(c) if the two variances are equal, the filter returns the
arithmetic mean value of the pixels in S xy .
41
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (3)
An adaptive expression for obtaining a filtered image
based on the assumptions:
fˆ ( x, y ) g ( x, y )
2
g
(
x
,
y
)
m
L
2
L
42
43
Adaptive Filters:
Adaptive Median Filters (1)
The notation:
zmin minimum intensity value in S xy
zmax maximum intensity value in S xy
zmed median intensity value in S xy
z xy intensity value at coordinates ( x, y )
S max maximum allowed size of S xy
44
Adaptive Filters:
Adaptive Median Filters (2)
The adaptive median-filtering works in two stages:
Stage A:
A1 = zmed zmin ;
A2 = zmed zmax
if A1>0 and A20 and B20 and A20 and B2 amplify noise !!!
64
Inverse Filtering
1. We can't exactly recover the undegraded image
because N (u, v) is not known.
2. If the degradation function has zero or very
small values, then the ratio N (u , v) / H (u, v) could
easily dominate the estimate F (u, v).
65
Inverse Filtering
EXAMPLE
The image in Fig. 5.25(b) was inverse filtered using the
exact inverse of the degradation function that generated
that image. That is, the degradation function is
H (u , v ) e
2
k u M /2 ( v N /2)2
5/6
, k 0.0025
66
The poor performance
of direct inverse
filtering in general
A Butterworth
lowpass
function of
order 10
67
Minimum Mean Square Error (Wiener)
Filtering
N. Wiener (1942)
Objective
Find an estimate of the uncorrupted image such that the mean
square error between them is minimized
2
ˆ
e E (f f)
2
68
Minimum Mean Square Error (Wiener)
Filtering
2
1
H
(
u
,
v
)
G (u, v)
Fˆ (u, v)
2
H (u, v) H (u, v) S (u, v) / S f (u, v)
H(u,v): degradation function
H*(u,v): complex conjugate of H(u,v)
|H(u,v)|2 = H(u,v)H*(u,v)
Sn(u,v)= |N(u,v)|2: power spectrum of the noise
Sf(u,v)= |F(u,v)|2 : power spectrum of the undegraded
image
69
Minimum Mean Square Error (Wiener)
Filtering
2
1
H
u
v
(
,
)
G(u, v)
Fˆ (u, v)
2
H (u, v) H (u, v) K
K is a specified constant. Generally, the value of K is
chosen interactively to yield the best visual results.
70
Minimum Mean Square Error (Wiener)
Filtering
71
Left:
degradated
image
Middle:
inverse
filtering
Right:
Wiener
filtering
72
Constrained Least Squares Filtering
• In Wiener filter, the power spectra of the undegraded
image and noise must be known. Although a constant
estimate is sometimes useful, it is not always suitable.
• Constrained least squares filtering just requires the mean
and variance of the noise.
73
Constrained Least Squares Filtering
H *(u, v)
F (u , v)
G (u, v)
2
2
| H (u, v) | | P(u, v) |
P(u , v) is the Fourier transform of the function
0 1 0
p ( x, y ) 1 4 1
0 1 0
is a parameter
(Regualarization term: ensure « smoothness » at high frequenciesTikhonov regularization - minimize the output of the Laplacian 74
operator ! )
Examples
75