LECTURE 5 Digital Image Restoration

  Digital Image Processing

Image Restoration

Image Restoration

• Image restoration vs. image enhancement

   Enhancement:

  

  largely a subjective process

  

  Priori knowledge about the degradation is not a must (sometimes no degradation is involved)

  

  Procedures are heuristic and take advantage of the psychophysical aspects of human visual system

   Restoration:

  

  more an objective process

  

  Images are degraded

  

  Tries to recover the images by using the knowledge about the degradation

Image Restoration

  Image restoration concerns the removal or reduction of degradations which have occurred during the acquisition of the image. Such degradations may include noise, which are errors in

the pixel values, or optical effects such as out of focus blurring, or

blurring due to camera motion. We shall see that some restoration

techniques can be performed very successfully using neighbourhood operations, while others require the use of frequency domain processes. Image restoration remains one of

the most important areas of image processing, but in this chapter

the emphasis will be on the techniques for dealing with restoration, rather than with the degradations themselves, or the properties of electronic equipment which give rise to image degradation.

• Two types of degradation _

  Additive noise

   Spatial domain restoration (denoising) techniques are preferred _

  Image blur

   Frequency domain methods are preferred

• We model the degradation process by a degradation function h(x,y), an additive noise term, (x,y), as g(x,y)=h(x,y)
• f(x,y)+ (x,y) _

  _ f(x,y) is the (input) image free from any degradation g(x,y) is the degraded image _{ *} is the convolution operator _ The goal is to obtain an estimate of f(x,y) according to the knowledge about the degradation function h and the additive noise  _

  In frequency domain: G(u,v)=H(u,v)F(u,v)+N(u,v)

• Three cases are considered in this Chapter _

  _ g(x,y)=f(x,y)+ (x,y) (5-2~5-4) g(x,y)=h(x,y)

• ^* f(x,y) (5-5~5-6) _ g(x,y)=h(x,y) * f(x,y)+ (x,y) (5-7~5-9)

  An Image Degradation Model

  

A Model of the Image Degradation/

Restoration Process

  

A Model of the Image Degradation/

Restoration Process

  Noise

Noise Model

• We first consider the degradation due to noise only _

  h is an impulse for now ( H is a constant)

• White noise _

  Autocorrelation function is an impulse function multiplied by a constant _{N  1 M  1}  a ( x , y ) ( s , t ) ( s x , t y ) N ( x , y )

  

    

  

    _{t  s }  It means there is no correlation between any two pixels in the noise image

   _ There is no way to predict the next noise value The spectrum of the autocorrelation function is a constant (white) (the statement in page 222 about white noise is wrong)

Gaussian Noise

• Noise (image) can be classified according the distribution of the values of pixels (of the noise image) or its (normalized) histogram
• Gaussian noise is characterized by two parameters, 

  2 (mean) and σ (variance), by _{2 2}

  1 ( z ) /

  2     p ( z ) e

  

  2  

• 70% values of z fall in the range [(-σ),(+σ)]
• 95% values of z fall in the range [(-2σ),(+

  2σ)]

  Gaussian Noise

  Gaussian Noise

  Gaussian Noise

Other Noise Models

• Rayleigh noise
₂

  2 _{( z a ) / b}     ( z a ) e for z a   p ( z )

    b 

   for z  a 

  The mean and variance of this density are given by b ( 4 ) _{2 }  a b /

  4 and    

     a and b can be obtained through mean and

  4 variance

Other Noise Models

• Erlang (Gamma) noise _{b b }
₁  a z _{ az} e for z

    p ( z ) 

   ( b 1 )!   for z

   

  The mean and variance of this density  are given by ₂ b  b / a and

  

   

₂

a and b can be obtained through

a mean and variance

Other Noise Models

• Exponential noise _{ az}

   ae for z  p ( z )

     for z

    

  The mean and variance of this density are given by  ₂

  1 Special case pf Erlang PDF 1 / a and     ₂

with b=1 a

Other Noise Models

• Uniform noise

  

1

  if a z b   p ( z )

   

b a

  





   otherwise  

  The mean and variance of this density are given by ₂

  ( b a ) ₂  ( a b ) / 2 and

  

   

  12

Other Noise Models

• Impulse (salt-and-pepper)

  noise P for z a _a 

   

   p ( z ) P for z b

    _b   otherwise 

   If either Pa or Pb is zero, the impulse noise is called unipolar

   a and b usually are extreme values because

impulse corruption is

usually large compared with the strength of the image signal

   It is the only type of noise that can be distinguished Salt-and-pepper Noise

  Salt-and-pepper Noise

  A Sample Image

  Effect of Adding Noise to Sample Image

  Effect of Adding Noise to Sample Image

Periodic Noise

• Arises typically from electrical or electromechanical

  interference during image

  acquisition
• It can be observed by visual inspection both in the spatial domain and frequency domain
• The only spatially dependent noise will be considered

  Periodic Noise

Noise Removal

  Salt and pepper noise, Gaussian noise and speckle noise can all be cleaned by using spatial filtering techniques.

Periodic noise, however, requires the use

of frequency domain filtering because whereas the other forms of noise can be modelled as local degradations, periodic noise is a global effect. Noise Removal

  Noise Removal

  Noise Removal

  Noise Removal

  Noise Removal

  Noise Removal

  Noise Removal

Estimation of Noise Parameters

• Periodic noise

   Parameters can be estimated by inspection of the spectrum

• Noise PDFs

   From sensor specifications

   If imaging sensors are available, capture a set of images of plain environments

   If only noisy images are available, parameters of the PDF involved can be estimated from small patches of constant regions of the noisy images

Estimation of Noise Parameters

• In most cases, only mean and variance are to be

  estimated

   Others can be obtained from the estimated mean and variance

• Assume a sub-image with plain scene is available

  and is denoted by S

  

  1 ˆ z ( x , y )

    i i

   N

  ( x , y )  S S i i

  

  1

  1

  2

  2 

  2 

  2 ˆ ( z ( x , y ) ) ( z ( x , y ) )

    

       i i 1 i i

   

  N N

  ( x , y ) S ( x , y ) S

  S i i S i i

  1

  2 

  2 ˆ ( z ( x , y ) )

      2 i i

   N

  1  ( x , y ) _S S i i

• Mean filters

  1 ) , ( ˆ mn S t s _{y x} t s g y x f

  1 ) , ( ) , (

  ) , ( ) , (

  S t s Q t s g t s g y x f _, ^,

   _{y x} ^{y x} S t s Q

     ^{ }

  1 ) , ( ˆ

  ) , ( ) , (

   _{y x} S t s t s g mn y x f _,

    _  _

     

     

  1 ) , ( _, ) , ( ) , ( ˆ

  ) , ( ) , (

   Arithmetic mean filter

  S t s t s g mn y x f _,

   _  _{y x}

   Q=0arithmetic mean filter

   Negative Q works for salt noise

   Positive Q works for pepper noise

   Q: order of the filter

   Contraharmonic mean filter

  

Works well for salt noise but fails for pepper noise

   Harmonic mean filter

   Tends to preserve more details

   Geometric mean filters

   S x,y is the mask

   g(x,y) is the corrupted image

  ) , ( ˆ Restoration in the Presence of Noise Only (De-Noising) Corrupted by Gaussian Noise

De-Noising

  Mean Filtering

  Geometric Mean Filtering Corrupted by pepper noise Corrupted by salt noise

De-Noising

  3x3 Contraharmonic Q=1.5

  3x3 Contraharmonic Q=-1.5

  De-Noising

Filters Based on Order Statistics (De-Noising)

  ˆ f ( x , y ) median { g ( s , t )}

  S (s,t) Median filter ^{x , y}

• 

   th

  Median represents the 50 percentile of a ranked set of numbers

• Max and min filter

   th

  Max filter uses the 100 percentile of a ranked set of numbers 

  Good for removing pepper noise 

  Min filter uses the 1 percentile of a ranked set of numbers 

  Good for removing salt noise

  Midpoint filter 

• 1

   ˆ f ( x , y ) max { g ( s , t )} min { g ( s , t )}

     ( s , t )  S ( s , t )  S  _{xy xy}

  2  

   Works best for noise with symmetric PDF like Gaussian or uniform noise

  Corrupted by salt & pepper noise One pass median filtering

De-Noising

  Two pass median filtering Three pass median filtering

  Corrupted by pepper noise

  Corrupted by salt noise

De-Noising

• Alpha-trimmed mean filter takes the mean value of the pixels enclosed by an m×n mask after deleting

  the pixels with the d/2 lowest and the d/2 highest

  gray-level values

   g r

  (s,t) represent the remaining mn-d pixels 

  It is useful in situations involving multiple types of noise like a combination of salt-and-pepper and Gaussian

   

  



_xy S t s r t s g d mn y x f

  ) , ( ) , (

  1 ) , ( ˆ

  Alpha-Trimmed Mean Filter (De-Noising)

De-Noising

  Corrupted by additive Uniform noise

  Added salt & pepper noise 5x5 Mean Filtering

  5x5 Geo-Mean Filtering

  5x5 Median Filtering

  5x5 Alpha- trimmed Mean Filtering

Adaptive Filters (De-Noising)

• Adaptive Local Noise Reduction Filter

  2 

  Assume the variance of the noise is either known 

   or can be estimated satisfactorily

   Filtering operation changes at different regions of an

  2 image according to local variance calculated within

   L an M×N region

  2

  2 

  If , the filtering operation is defined as   

  L 

  2



   ˆ f ( x , y ) g ( x , y ) [ g ( x , y ) m ]

     L

  2



  L

  2

  2 

  If , the output takes the mean value   

  L 

  2 

   

  That is: is set to be

  

1

  2 

  L

  2

  2 

     L

   At edges, it is assumes that Corrupted by Gaussian noise

  Mean Filtering

De-Noising

  Geo-Mean Filtering

  Adaptive Filtering

• Median filter is effective for removing salt-and- pepper noise

   The density of the impulse noise can not be too large

• Adaptive median filter

   Notation

   Z min : minimum gray value in S xy

   Z _max : maximum gray value in S _xy

   Z _med : median of gray levels in S _xy

   Z xy : gray value of the image at (x,y)

   S _max : maximum allowed size of S _xy

  Adaptive Median Filter (De-Noising)

Adaptive Median Filter (De-Noising)

• Two levels of operations

   Level A:

  Used to test whether

  

  A1= Z –Z _{med min} Z is part of s-and-p _med

  

  A2= Z med –Z max noise. If yes, window

  

  size is increased If A1 > 0 AND A2 < 0, Go to level B else increase the window size by 2

  

  If window size <= S max repeat level A else output Z _xy

   Level B:

  Used to test whether

  

  Z is part of s-and-p B1= Z –Z _{xy min xy}

  

  noise. If yes, apply B2= Z –Z _{xy max} regular median filtering

  

  If B1 > 0 AND B2 < 0, output Z xy else output Z _med

  Median Filtering Adaptive Median

De-Noising

  Filtering

Periodic Noise Reduction by Frequency Domain Filtering

• Lowpass and highpass filters for image enhancement have been studied
• Bandreject, bandpass, and notch filters as tools for periodic noise reduction or removal are to be studied in this section.

Bandreject Filters

• Bandreject filters remove or attenuate a band of frequencies about the origin of the Fourier transform.
• Similar to those LPFs and HPFs studied, we can construct ideal, Butterworth, and Gaussian bandreject filters

Bandreject Filters

• Gaussian bandreject filter

     

     

     

  W v u D D v u D H e v u

  ) , ( ) , (

  2

  1

  2

  2 , 1 ) (

  Bandreject Filters

Notch Filters

      v u

  Notch filters that pass, rather than suppress: ) , ( , 1 ) ( v u H v u H nr np

• NR filters become highpass filters if
• NP filters become lowpass filters if

    v u

Notch Filters

  You can see the effect of scan lines Notch pass

  Spectrum of filter image

  IFT of NP Result of filtered image NR filter

  Optimum Notch Filtering

Optimum Notch Filtering

• In the ideal case, the original image can be restored if the noise can be estimated completely.

   That is: f ( x , y ) g ( x , y ) ( x , y )

    

• However, the noise can be only partially estimated. This means the restored image is not exact.

   ˆ

  Which means ˆ f ( x , y ) g ( x , y ) ( x , y )

     ˆ ( x , y )

  IFT H ( u , v ) G ( u , v ) 

    

Optimum Notch Filtering

• In this section, we try to improve the restored image by introducing a modulation function _

  ˆ ˆ f ( x , y ) g ( x , y ) w ( x , y ) ( x , y ) _   

  Here the modulation function is a constant within a _ neighborhood of size (2a+1) by (2b+1) about a point (x,y) We optimize its performance by minimizing the local variance of the restored image at the position (x,y)

  2 a b

  1

  2  ˆ ˆ  ( x , y ) f ( x s , y t ) f ( x , y )

      

    

     (

  2 a 1 )( 2 b 1 )   s a t b     a b

  1 ˆ ˆ f ( x , y ) f ( x s , y t )

    

    ( 2 a 1 )( 2 b 1 )

  s a t b

     

Optimum Notch Filtering

  Points on or near Edge of the image can be treated by considering partial neighborhoods a b

  1

  2 ( x , y )

  {[ g ( x s , y t )   

    

  ( 2 a 1 )( 2 b

1 )

  s   a t   b

  ˆ w ( x s , y t ) ( x s , y t )]

       

  2 ˆ [ g ( x , y ) w ( x , y ) ( x , y ]}

    

  Assumption:

  w ( x s , y t ) w ( x , y ) for a s a and b t b         

  ˆ ˆ w ( x , y ) ( x , y ) w ( x , y ) ( x , y )

     

Optimum Notch Filtering

    

  ) , ( ² y x 

    To minimize

       

  2 y x y x y x y x g y x y x g y x w

  2

  ) , ( ) , ( ˆ ) , ( ) , (

  ) , (

ˆ

) , ( ˆ

      ) , ( ˆ

  ) , ( ) , (

  2  

      

  1 ) , ( y x y x w y x g t y s x t y s x w t y s x g b a y x a a s b b t

  

1

2 )( 1 2 (

  ) , ( ) , ( {[ )

  ) , ( ) , ( [ )] , ( ˆ

  2 )} , ( ˆ

  2

  

  

y x w

y x

          

Optimum Notch Filtering

  ) , ( ˆ ) , ( ) , ( ) , ( ˆ y x y x w y x g y x f

     ) , ( y x w

) , (

  ˆ

y x

   ) , ( y x g

  Optimum Notch Filtering

  Optimum Notch Filtering

Optimum Notch Filtering

  ˆ g ( y x , ) f ( x , y )

  Image size: 512x512 a=b=15

Linear, Position-Invariant Degradation

• Degradation Model

  ) , ( )] , ( [ ) , (

H y x y x f y x g

    

  )] , ( [ )] , ( [ )] , ( ) , ( [

  2

  1

  2

     ) , ( )] , ( [ )] , ( [ ) , (

            

  H y x g y x f y x f H y x g In the absence of additive noise:

  For scalar values of a and b, H is linear if: H is Position-Invariant if:

Linear, Position-Invariant Degradation In the presence of additive noise:

    g ( x , y ) f ( , ) h ( x , y ) d d ( x , y )

            

       g ( x , ) h ( x , y ) f ( x , y ) * y

  ( x , y )   

  G ( u , v ) H ( u , v ) F ( u , v ) N ( u , v )  

• • Many types of degradation can be approximated by

  linear, position-invariant processes

• • Extensive tools of linear system theory are available

• • In this situation, restoration is image deconvolution

• Principal way to estimate the degradation function for use in image restoration: _

  Observation _ Experimentation _ Mathematical modeling Estimating the Degradation Function

Estimating by Image Observation

• We look for a small section of the image that has

  g ( y x , ) strong signal content ( ) and then s

construct an un-degradation of this section by

  ˆ using sample gray levels ( ). f ( x , y ) s

  G ( u , v ) s

  H ( u , v )  s

  ˆ F ( u , v ) s

  Now, we construct a function on a large scale, H ( v u , ) but having the same shape.

Estimating by Experimentation

• We try to obtain impulse response of the degradation by imaging an impulse (small dot of light) using the system. Therefore

  G ( u , v ) H ( u , v ) 

  A

  Estimating by Modeling _{2 2 5 / 6} k ( u v )

    Atmospheric turbulence model:

  H ( u , v ) e 

  High turbulence k=0.0025

  Negligible turbulence Low

  Mid turbulence turbulence k=0.00025 k=0.001

Estimating by Modeling

  Blurring by linear motion: T g ( x , y ) f [ x x ( t ), y y ( t )] dt

     

  T _{  } j 2 [ ux ( t ) vy ( t )] G ( u , v ) F ( u , v ) e dt

   

  T _{ j} 2 [ ux ( t )  vy ( t )] _

  H ( u , v ) e dt  

   T _{ } 2 uat / T

  H ( u , v ) e dt if x ( t ) at / T and y ( t )

     

   T _{ } j ua sin( ua ) e   ua

  

Estimating by Modeling

  if x ( t ) at / T and y ( t ) bt / T 

    T j ( ua vb )

     H ( u , v ) sin[ ( ua vb )] e

  

  

( ua vb )  

Inverse Filtering

  The simplest approach to restoration is direct inverse filtering: G ( u , v ) Even if we know the

  ˆ F ( u , v )

  

  degradation function, we

  H ( u , v )

  cannot recover the un- degraded image

  N ( u , v ) ˆ

  F ( u , v ) F ( u , v )  

  H ( u , v )

  If the degradation has zero or very small values, then the ratio N/H could easily dominate our estimation of F .

  One approach to get around the zero or small-value problem is to limit the filter frequencies to value near the origin.

  Degraded Image

Inverse Filtering

  Filtering with H cut off outside a radius of

  40 Full inverse

  Filtering Filtering with H cut off outside a radius of

  cut off outside a

Minimum Mean Square Error Filtering (Wiener Filtering)

  ] ) ˆ [(

  2

    This approach incorporate both the degradation

function and statistical characteristic of noise into the

restoration process.

  Image and noise are random process

  

The objective is to find an estimation for f such that minimized e

  2

Wiener Filtering

  Full inverse filtering Radially limited inverse filtering

  Wiener filtering

  (K was chosen interactively) Inverse filtering Reduced noise variance

  Wiener filtering

Wiener Filtering

Geometric Transformations

• Unlike the techniques discussed so far, geometric transformations modify the spatial relationships between pixels in an image.

Geometric transformation: RUBBER-SHEET TRANSFORMATION Basic Operations:

  1. Spatial Transformation

  2. Gray-level Interpolation

Spatial Transformations

  x  r ( x , y )  y  s ( x , y )

   x  r ( x , y ) c x c y c xy c

      

  

1

  2

  3

  4 y  s ( x , y ) c x c y c xy c

      

  5

  6

  7

  8

  Spatial Transformations

Gray-level Interpolation

  

d y x c y b x a y x v

  

     

   ) , (

  

Geometric Transformations