Remote Sensing Image Fusion Scheme using Directional Vector in NSCT Domain

(1)

DOI:_{10.12928/TELKOMNIKA.v14i1.2747} ₅₉₈

Remote Sensing Image Fusion Scheme using

Directional Vector in NSCT Domain

Baohui Tian1*, Lan Lan1, Hailiang Shi2, Yunxia Pei2 1

Department of Communication & Information Engineering,

Henan Vocational & Technical College of Communications, Zhengzhou 450005, Henan, China 2

College of Mathematics & Information Science, Zhengzhou University of Light Industry, Zhengzhou 450002, Henan, China

*Corresponding author, e-mail: 18341615@qq.com

Abstract

A novel remote sensing image fusion scheme is presented for panchromatic and multispectral images, which is based on NonSubsampled Contourlet Transform (NSCT) and Principal Component Analysis (PCA). The fusion principles of the different subband coefficients obtained by the NSCT decomposition are discussed in detail. A PCA-based weighted average principle is presented for the lowpass subbands, and a selection principle based on the variance of the directional vector is presented for the bandpass directional subbands, in which the directional vector is assembled by the NSCT coefficients of the different directional subbands but the same coordinate. The proposed scheme is tested on two sets of remote sensing images and compared with some traditional multiscale transform-based image fusion methods, such as discrete wavelet transform, stationary wavelet transform, dual-tree complex wavelet transform, contourlet transform. Experimental results demonstrate that the proposed scheme provides superior fused image in terms of several relevant quantitative fusion evaluation indexes. Keywords: Image Fusion, Remote Sensing, Nonsubsampled Contourlet Transform, Principal Eigenvector,

Directional Vector

1. Introduction

The field of remote sensing is a continuously growing market with applications like vegetation mapping and observation of the environment. The increase in applications is due to the availability of high quality images for a reasonable price and improved computation power. However, as a result of the demand from higher classification accuracy and the need in enhanced positioning precision there is always a need to improve the spectral and spatial resolution of remote sense images. These requirements can be either fulfilled by building new satellites with a superior resolution power, or by the utilization of image fusion techniques. The main advantages of the second alternative is the significantly lower expense [1].

In remote sensing systems, panchromatic (PAN) images of high spatial resolution can provide detailed geometric information, such as shapes, features and structures of objects of the earth’s surface, and multispectral (MS) image with relatively lower spatial resolution are used to obtain spectral information necessary for environmental applications. The different objects within images of high spectral resolution and high spatial resolution are easily identified [2]. The goal of image fusion is to obtain a high spatial resolution multispectral image which combines the spectral characteristic of the low spatial resolution data with the spatial information of the panchromatic images. Unlike other application, e.g. image fusion in military missions or computer-aided quality control, the main constraint in remote sensing is to preserve the spectral information for tasks [1, 2].

In the last decades, many methods have been proposed for fusing panchromatic and multispectral images. The well known methods are those based on the intensity-hue-saturation transform (IHS), principal component analysis (PCA), discrete wavelet transform (DWT) and so on [3]. Unfortunately, these methods have their limitations. The main drawback of the methods based on IHS or PCA is the high distortion of the original spectral information [4]. The two dimensional (2-D) DWT is good at isolating the discontinuities at edge points, but cannot effectively represent the ‘line’ or the ‘curve’ discontinuities properly [5]. In addition, DWT can

(2)

capture only limited directional information, and thus cannot represent the directions of the edges accurately [6].

In recent years, Do and Vetterli developed a true two dimentsional image representation method, the Contourlet Transform (CT), which is achieved by combining the Lapacian Pyramid (LP) and the Directional Filter Bank (DFB). Compared with the traditional DWT, CT is not only with multiscale and localization, but also with multidirection and anisotropy. As a result, CT can represent edges and other singularities along curves much more efficiently. However, CT lacks the shift-invariance, which is desireable in many image applications such as image enhancement, image denoising and image fusion. In 2006, Cunha et al. proposed an overcomplete transform, the NonSubsampled Contourlet Transform (NSCT), which inherits the perfect properties of CT, and possesses the shift-invariance. When the NSCT is introduced into image fusion, more information for fusion can be preserved and the impacts of mis-registration on the fused results can also be reduced effectively. Therefore, the NSCT is more suitable for image fusion [6].

In this paper, a novel image fusion scheme using NSCT for panchromatic and multispectral images is proposed. The outline of this paper is as follows: Section 2 briefly reviews the NSCT theory. Section 3 describes the fusion scheme in detail. Experimental results and discussion are presented in Section 4, where the proposed fusion scheme is compared with some traditional. Lastly comes the conclusion.

2. NSCT

The NSCT is a shift-invariance version of the CT. The CT employs the LP for multiscale decomposition, and the DFB for directional decomposition. In order to get rid of the frequency aliasing and enhance directional selectivity and shift-invariance, the NSCT eliminates the downsampling and the upsampling during the decomposition and the reconstruction of an image; instead it is built upon coupling NonSubsampled Pyramid (NSP) with the NonSubsampled Directional Filter Bank (NSDFB). The NSCT provides not only multiscale analysis but also geometric and directional representation [7].

The core of the NSCT is the filter design in the nonseparable two-channel NSP and NSDFB. The goal is to design filters satisfying the Bezout identity (Perfect Reconstruction, PR) and enforcing other properties such as sharp frequency response, easy implementation, regularity of the frame elements, and the tightness of the corresponding frames. Of course, if the tightness constraint is relaxed, the FIR filters with linear phrase can also be obtained. In the NSCT, the mapping approach is adopted to design the two-channel 2-D filter banks with PR. In addition, the filters designed with the mapping approach are factored into a ladder or lifting structure to simply computation. Therefore, the NSCT is also with a fast implementation. More detail can be seen in [7].

The NSP, employed by the NSCT for the multiscale property, is a two-channel NSFB. The NSP is completely different from the Laplacian Pyramid (LP) in the CT, because it has no downsampling or upsampling. To achieve the multiscale decomposition, the expansion is conceptually similar to the 1-D à trous and has J1 redundancy, where J denotes the number of decomposition levels. The ideal frequency support of the lowpass filter at j-th stage is the

region 2

[ ( / 2 ), ( / 2 )] j  j _{. Accordingly, the ideal support of the equivalent highpass filter is the}

complement of the lowpass on the 1 1 2 2

[(/ 2j ), ( / 2 j)] \ [( / 2 ), ( / 2 )]j  j region. The filters for subsequent stages are obtained by upsampling the filters of the first stage with the sampling matrix. The structure is thus different from the tensor product à trous. In particular, one bandpass image is produced at each stage resulting in J1 redundancy.

The NSDFB is a shift-invariance version of the critically sampled DFB in the CT. It is constructed by eliminating the downsampling and the upsampling in the DFB. The building block of the NSDFB is also a two-channel NSFB. However, the ideal frequency response for the NSDFB is different. The upsampling of all filters is based on quincunx matrix. The higher level decompositions follow the similar strategy.

The NSCT is flexible in that it allows any number of 2l directions in each scale, where l denotes a positive integer. In particular, it can satisfy the anisotropic scaling law and has

(3)

redundancy given by

1 J 2lj

j





, where l_j denotes the number of direction decomposition levels in NSDFB at the j-th scale.

(a) Panchromatic image (b) Multispectral image

(e) Fused image using DTCWT-based method

(f) Fused image using CT-based method

(g) Fused image using NSCT-based method (h) Fused image using the proposed scheme Figure 1. Source images and the fused images (256*256 pixels)

(4)

3. Proposed Fusion Scheme

There are three basic steps for image fusion [8] using NSCT:

Firstly, the source images are decomposed into NSCT representations with different resolutions and directions. To achieve this, we should perform IHS transform on multispectral image to obtain the corresponding grayscale intensity components i MS( ) firstly. To reduce the computational complexity, the grayscale intensity component can be obtained by averaging the three component of the MS image.

( ) ( ) / 3

i MS  R G B (1)

Then perform J-level NSCT on PAN and i MS( ) to obtain the corresponding coefficients.

{ L ( , ), H ( , , , ) (1 , 1, 2, , 2 )}lj

PAN PAN

C m n C m n j k  j J k  (2)

And,

{ L ( , ), H ( , , , ) (1 , 1, 2, , 2 )}lj

MS MS

C m n C m n j k  j J k  (3)

Secondly, these NSCT representations are composited according to some fusion principles to form the fused image’s representation.

{ L( , ), H( , , , ) (1 , 1, 2, , 2 )}lj

F F

C m n C m n j k  j J k  (4)

The fusion principles will be discussed in detail in the following sections.

Thirdly, the fused image is reconstructed using the inverse NSCT with the above composited NSCT coefficients.

3.1. Lowpass Subband Fusion Principle

Generally, the lowpass subband coefficients of the fused image can be simply acquired by averaging the lowpass subband coefficients of the source images. However, this will reduce the fused image contrast. There is another way to fuse the lowpass subband coefficients by selecting the PAN lowpass subband coefficients, which may result in high distortion of the original spectral information. In this paper, a PCA-based weighted average fusion principle lowpass subband coefficients is proposed.

The fusion principle is, firstly apply PCA to C_PANL ( , )m n and C_MSL ( , )m n at the highest transform scale to obtain the principal eigenvector, which is corresponding to the larger eigenvalue, and then get the composited coefficients by:

1 2 1 2

( , ) ( ( , ) ( , )) / ( )

L L L

F PAN MS

C m n  w C m n w C m n w w (5)

Where w₁ and w₂ are the elements of the principal eigenvector.

In practice, we have to convert the 2-D subbands C_PANL ( , )m n and C_MSL ( , )m n into 1-D vectors by simply stacking each subband column-by-column so that the principal component can be computed.

To show the superiority of applying PCA to fuse the lowpass subbands, two examples are given below. i) For the “disk” multifocus images, w₁=0.6859 (48.52%) and w₂=0.7277 (51.48%), and ii) For remote sensing images as shown in Fig. 1(a)(b), w₁=0.6944 (49.11%) and

w =0.7196 (50.89%), where the normalized weights are listed inside the parentheses. It is clear that, if two input images are equally important like in the disk images, two approximately equal weights were assigned to both images.

(5)

3.2. Bandpass Subbands Fusion Principle

Image’s important features in a local region, including textures, edges, lines, etc., generally have some characteristics such as multiscale and strong direction. In the NSCT domain, such features correspond to some NSCT coefficients with large magnitude and these coefficients are distributed only in a few directions [6]. The noise is different from the above, probably it also corresponds to some NSCT coefficients with large magnitude, but the local orientation energy of noise distributes uniformly in all directions. To distinguish the image features from the noise properly, and to measure the image pixel importance, the directional vector [6] in the NSCT domain is introduced, and then the variance of directional vector follows.

The directional vector of an image I at the j-th scale and coordinate ( , )m n is defined as:

( , , ) ( ( , , ,1), ( , , , 2), , ( , , , 2 ))lj

H H H H

I I I I

DV m n j  C m n j C m n j C m n j (6)

Where l_j denotes the direction decomposition level at the j-th scale. The directional vector is used to represent the distribution of the image feature in each direction.

The variance of the directional vector is computed by: 2 2 1 1 ( , , ) ( ( , , , ) ( , , )) 2 l j j H H

I l I I

V m n j C m n j k M m n j





 (7)

Where 2

1 1 ( , , ) ( , , , ) 2 l j j H H

I l I

M m n k 



C m n j k .

When the two or more corresponding directional vector variances are compared, the greater one is most probably correspond to the important feature, and the rest are correspond to the noise.

Then the fusion principle for the bandpass directional subband coefficients can be defined as:

( , , , ), ( , , ) ( , , ) ( , , , )

( , , , ), ( , , ) ( , , )

H H H

H PAN PAN MS

F H H H

MS PAN MS

C m n j k V m n j V m n j C m n j k

C m n j k V m n j V m n j

 



  _

 (8)

The above selection principle can not only extract more useful geometric detail information effectively, but also avoid the noise being transferred into the fused image.

4. Experimental Results

The proposed method is compared with some well-known methods based on multiscale transform including DWT, Stationary Wavelet Transform (SWT), Dual-Tree Complex Wavelet Transform (DTCWT), CT, and NSCT. For these comparative methods, the lowpass subband coefficients are simply merged by the average principle, and the bandpass subband coefficients are simply merged by the ‘absolute maximum choosing’ principle. Three levels decomposition, together with ‘9-7’ wavelet are used in the DWT, SWT, DTCWT, and three levels decomposition with 4, 8, 16 directions from coarser scale to finer scale are used in the CT and NSCT. Meanwhile, in the proposed scheme the decomposition level of the NSCT is 1 for less computational complexity, with 8 directions.

In this paper, several objective evaluation measures, such as mutual information, correlation coefficient and so on, which have been proved to be validated in large degree, are considered to quantitatively evaluate the fusion performance.

(1) Mutual Information (MI) [8, 9] is a metric defined as the sum of mutual information between each source image and the fused image. Considering the source image A and the fused image F, the mutual information between them is calculated by:

( , ) ( , ) log

( ) ( ) AF AF AF

a f A F

p a f

MI p a f

p a p f

(6)

Where p_AF is the jointly normalized histogram of A and F, p_A and p_F are the normalized histogram of A and F, respectively. Obviously, larger MI value indicates better fusion result. Note that the mutual information used in this paper is calculated by the corresponding bands of the fused image and the multispectral image.

(2) Correlation Coefficient (CC) [2] denotes the degree of the correlation of each bands of the fused image and the MS, and it should be as close to 1 as possible.

1 1

2 2

1 1 1 1

( ( , ) )( ( , ) ) ( , )

( ( ( , ) ) )( ( ( , ) ) )

M N i j

M N M N

i j i j

X i j X Y i j Y C C X Y

X i j X Y i j Y

          

 

(10)

(3) Peak Signal Noise Rate (PSNR) [10] between two bands of the fused image and the multispectral image. 2 2 1 1 255 10 lg ( ( , ) ( , )) M N i j M N P SN R

R i j F i j

 

 



 

(11)

(4) Universal Image Quality Index (UIQI) [11] is designed by modeling any image distortion as a combination of three factors: loss of correlation, luminance distortion and contrast distortion. It is mathematically defined by:

2 2 2 2

2 2

xy x y

x y x y

x y UIQI x y          

  (12)

Where x y, are weighted-based image signals centered at p, x y, denote the mean of x y, , , ,

x y xy

   denote the variance of x y, and covariance of x y, . Its dynamic range is [ 1,1] , and the best value is achieved if and only ifyx.

(5) The Root Mean Square Error (RMSE) [12] measures the difference between the fused and the original image for each band. Its ideal value is 0.

2₍ ₎ 2₍ ₎ 2₍ ₎

i i i

RMSE B bias B SD B (13)

Where bias means the difference between the means of the fused and the original image and SD means the standard deviation of the difference image, relative to the mean of the original image.

(6) Relative Average Spectral Error index (RASE) [12]. It characterizes the average performance of the method in the spectral bands considered and defined as:

2 1

1 0 0 1

( )

i i

R A S E R M S E B

M N 





(14)

Where M is the mean radiance of the N spectral bands B_i of the multispectral image. (7) Relative global dimensional synthesis error index (ERGAS) [13]. It is defined as:

2 2 1 ( ) 1 100 N i i i

RM SE B

h ERG AS

l N  M

 

 _ _

 



(15)

Where h is the resolution of the high spatial resolution image and l the resolution of the low spatial resolution image and M_i the mean radiance of each spectral band involved in the fusion. The lower the value of the RASE and ERGAS indexes, the higher the spectral quality of the fused image, their ideal values are both 0. Two sets of images with perfect registration are used to evaluate the proposed fusion scheme. The first set of the test images is shown in Figure

(7)

1(a), (b). The fused images with different fusion methods based on DWT, SWT, DTCWT, CT, NSCT and the proposed scheme are shown in Figure 1 (c)-(h).

(a) Panchromatic image (b) Multispectral image

(e) Fused image using DTCWT-based method (f) Fused image using CT-based method

(g) Fused image using NSCT-based method (h) Fused image using the proposed scheme Figure 2. Source images and the fused images (512*512 pixels)

From the human visual system, we can see that all the fusion methods are effective since they have equivalent fusion performance in some degree; the detail spatial information is well transferred into the fused images. The objective evaluations on the fused results are listed

(8)

in Table 1. The best results are indicated in bold. From Table 1, we can see that the proposed scheme takes all the best objective evaluations, which is obviously better than the other methods.

Table 1. The objective evaluations of Fig.1(c)~(h)

Methods Band MI CC PSNR UIQI RMSE RASE ERGAS

DWT

R 0.7586 0.7763 65.3668 0.4821 0.1279

49.8701 20.3074 G 0.7177 0.6942 64.3869 0.4801 0.1539

B 0.7275 0.7557 65.9968 0.4819 0.1375 SWT

R 0.9426 0.8735 68.1332 0.5367 0.1000

42.1649 14.5158 G 0.8989 0.8334 67.5575 0.5390 0.1068

B 0.9437 0.8673 68.7050 0.5410 0.0936 DTCWT

R 0.9092 0.8635 68.4625 0.4936 0.1098

42.7707 14.9359 G 0.8683 0.8210 67.3205 0.4942 0.1098

B 0.9086 0.8583 67.8679 0.5015 0.0963 CT

R 0.7328 0.7611 65.4316 0.4821 0.1364

49.6298 20.1121 G 0.7175 0.6991 64.4754 0.4793 0.1523

B 0.7604 0.7796 66.0960 0.4836 0.1264 NSCT

R 0.9498 0.8735 67.8963 0.5441 0.1027

42.8416 14.9854 G 0.9089 0.8316 67.2530 0.5453 0.1106

B 0.9408 0.8624 68.4183 0.5412 0.0967 Proposed

R 1.1802 0.9195 70.5419 0.6649 0.0758

37.2446 11.3254

G 1.1360 0.8852 69.7726 0.6679 0.0828

B 1.1848 0.9107 70.5144 0.6659 0.0760

In the second set experiment, a couple of Quickbird panchromatic and Quickbird multispectral (band 3, 2, 1) images covering the land area near the Pyramids, are used as PAN and MS test images, shown in Figure 2(a), (b). Similarly, the objective evaluations of various fusion methods are listed in Table 2. And similar conclusion that, the proposed scheme provides superior fused image in terms of the above quantitative fusion evaluation indexes, can be concluded from the fusion results.

Table 2. The objective evaluations of Figure 2(c)~(h)

Method Band MI CC PSNR UIQI RMSE RASE ERGAS

DWT

R 0.9673 0.8918 66.6282 0.6086 0.1189

29.0186 10.0380 G 0.9630 0.8849 66.5295 0.6192 0.1202

B 0.9462 0.8849 66.3133 0.6007 0.1233 SWT

R 1.4253 0.9583 71.0369 0.6627 0.0716

22.5566 6.0651 G 1.4297 0.9566 70.9183 0.6745 0.0725

B 1.3989 0.9532 70.6476 0.6584 0.0748 DTCWT

R 1.3734 0.9544 70.7459 0.6059 0.0740

22.9493 6.2781 G 1.3785 0.9525 70.6176 0.6205 0.0751

B 1.3470 0.9484 70.3403 0.6021 0.0775 CT

R 0.9666 0.9002 66.7134 0.5959 0.1177

28.8941 9.9521 G 0.9595 0.8946 66.6071 0.6078 0.1192

B 0.9392 0.8872 66.3755 0.5900 0.1224 NSCT

R 1.4320 0.9582 71.1138 0.6873 0.0709

22.4258 5.9950 G 1.4342 0.9567 71.0268 0.6982 0.0716

B 1.4008 0.9534 70.7642 0.6815 0.0738 Proposed

R 1.5193 0.9674 73.5420 0.6921 0.0536

19.5103 4.5374 G 1.5048 0.9656 73.3935 0.7024 0.0546

B 1.4665 0.9629 73.2280 0.6901 0.0556

In addition, in the experiments, we noticed that the superior fusion performance of the proposed scheme is at the cost of increasing computational complexity during the fusion process. In other words, the proposed scheme is time-consuming, and the computation load is heavier than the other fusion methods (except the traditional NSCT-based method). It takes about 60 minutes to fuse a pair of 512×512 pixels images on our PC. Similar phenomenon happens in the NSCT-based fusion methods, which is mainly due to the high redundancy of the NSCT.

(9)

4. Conclusion

This paper presents a novel remote sensing image fusion scheme for panchromatic and multispectral images, which is based on NSCT and PCA. In the fusion process of NSCT lowpass subbands coefficients of NSCT, a PCA-based weighted average principle is presented, in which the weights are the elements of the principal eigenvector. In the fusion process of bandpass subbands, a selection principle based on the variance of the directional vector is presented, in which the directional vector is assembled by the NSCT coefficients of the different directional subbands but the same coordinate. The experimental results demonstrate that the proposed scheme provides superior fused image in terms of the relevant quantitative fusion evaluation indexes.

Acknowledgements

This work has been supported by National Natural Science Foundation of China (No. 11501527) and Natural Science Research Program of Henan Educational Committee (No. 14A120012).

References

[1] T Bretschneider, O Kao. Image fusion in remote sensing. Proceedings of the 1st Online Symposium of Electronic Engineers. 2000.

[2] X Yang, L Jiao. Fusion algorithm for remote sensing images based on nonsubsampled contourlet transforms. Acta Automatica Sinica. 2008; 34(3): 274-281.

[3] Yong Chen, Jie Xiong, Huanlin Liu, Qiang Fan. Combine target extraction and enhancement methods to fuse infrared and LLL images. TELKOMNIKA Telecommunication, Computing, Electronics and Control. 2014; 12(3): 605-612.

[4] MC Audícana, JL Saleta, RG Catalán, R García. Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Transactions on Geoscience and Remote Sensing. 2004; 42(6): 1291-1299.

[5] Do MN, Vetterli M. The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 2005; 14(12): 2091-2106.

[6] Q Zhang, B Guo. Multifocus image fusion using nonsubsampled contourlet transforms. Signal Process. 2009; 89(7): 1334-1346.

[7] Cunha AL, J Zhou, Do MN. The nonsubsampled contourlet transform: theory, design and applications. IEEE Trans. Image Proecess. 2006; 15(10): 3089-3101.

[8] B Yang, S Li. Pixel-level image fusion with simultaneous orthogonal matching pursuit. Information Fusion. 2012; 13(1): 10-19.

[9] Yongxin Zhang, Li Chen, Zhihua Zhao, et al. A novel multi-focus image fusion method based on non-negative matrix factorization. TELKOMNIKA Telecommunication, Computing, Electronics and Control. 2014; 12(2): 379-388.

[10] Z Zhang, Blum RS. A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application. Proc. IEEE. 1999; 87(8): 469-472.

[11] Zhou Wang, Alan C Bovik. A universal image quality index. IEEE Signal Processing Letters. 2002; 9(3): 81-84.

[12] L Wald. Quality of high resolution synthesized images: Is there a simple criterion?. Proc. Int. Conf. Fusion Earth Data. 2000: 99-105.

[13] T Ranchin, L Wald. Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation. Photogramm. Eng. Remote Sens. 2000; 66: 49-61.

(1)

3. Proposed Fusion Scheme

There are three basic steps for image fusion [8] using NSCT:

( ) ( ) / 3

i MS  R G B (1)

Then perform J-level NSCT on PAN and i MS( ) to obtain the corresponding coefficients. { L ( , ), H ( , , , ) (1 , 1, 2, , 2 )}lj

PAN PAN

C m n C m n j k  j J k  (2)

And,

{ L ( , ), H ( , , , ) (1 , 1, 2, , 2 )}lj

MS MS

C m n C m n j k  j J k  (3)

Secondly, these NSCT representations are composited according to some fusion principles to form the fused image’s representation.

{ L( , ), H( , , , ) (1 , 1, 2, , 2 )}lj

F F

C m n C m n j k  j J k  (4)

The fusion principles will be discussed in detail in the following sections.

Thirdly, the fused image is reconstructed using the inverse NSCT with the above composited NSCT coefficients.

3.1. Lowpass Subband Fusion Principle

1 2 1 2

( , ) ( ( , ) ( , )) / ( )

L L L

F PAN MS

C m n  w C m n w C m n w w (5)

Where w₁ and w₂ are the elements of the principal eigenvector.

(2)

3.2. Bandpass Subbands Fusion Principle

The directional vector of an image I at the j-th scale and coordinate ( , )m n is defined as:

( , , ) ( ( , , ,1), ( , , , 2), , ( , , , 2 ))lj

H H H H

I I I I

DV m n j  C m n j C m n j C m n j (6)

Where l_j denotes the direction decomposition level at the j-th scale. The directional vector is used to represent the distribution of the image feature in each direction.

The variance of the directional vector is computed by: 2

2 1

( , , ) ( ( , , , ) ( , , )) 2

l j

H H

I l I I

V m n j C m n j k M m n j





 (7)

Where 2

1 1

( , , ) ( , , , ) 2

l j

H H

I l I

M m n k 



C m n j k .

When the two or more corresponding directional vector variances are compared, the greater one is most probably correspond to the important feature, and the rest are correspond to the noise.

Then the fusion principle for the bandpass directional subband coefficients can be defined as:

( , , , ), ( , , ) ( , , ) ( , , , )

( , , , ), ( , , ) ( , , )

H H H

H PAN PAN MS

F H H H

MS PAN MS

C m n j k V m n j V m n j C m n j k

C m n j k V m n j V m n j

 



  _

 (8)

The above selection principle can not only extract more useful geometric detail information effectively, but also avoid the noise being transferred into the fused image.

4. Experimental Results

( , ) ( , ) log

( ) ( )

AF AF

a f A F

p a f

MI p a f

p a p f

(3)

(2) Correlation Coefficient (CC) [2] denotes the degree of the correlation of each bands of the fused image and the MS, and it should be as close to 1 as possible.

1 1

2 2

1 1 1 1

( ( , ) )( ( , ) )

( , )

( ( ( , ) ) )( ( ( , ) ) )

M N

i j

M N M N

i j i j

X i j X Y i j Y

C C X Y

X i j X Y i j Y

          

 

(10)

(3) Peak Signal Noise Rate (PSNR) [10] between two bands of the fused image and the multispectral image. 2 2 1 1 255 10 lg ( ( , ) ( , )) M N i j M N P SN R

R i j F i j

 

 



 

(11)

2 2 2 2

2 2

xy x y

x y x y

x y UIQI x y          

  (12)

Where x y, are weighted-based image signals centered at p, x y, denote the mean of x y, , , ,

x y xy

   denote the variance of x y, and covariance of x y, . Its dynamic range is [ 1,1] , and the best value is achieved if and only ifyx.

(5) The Root Mean Square Error (RMSE) [12] measures the difference between the fused and the original image for each band. Its ideal value is 0.

2₍ ₎ 2₍ ₎ 2₍ ₎

i i i

RMSE B bias B SD B (13)

Where bias means the difference between the means of the fused and the original image and SD means the standard deviation of the difference image, relative to the mean of the original image.

(6) Relative Average Spectral Error index (RASE) [12]. It characterizes the average performance of the method in the spectral bands considered and defined as:

2 1

1 0 0 1

( )

i i

R A S E R M S E B M N 





(14)

Where M is the mean radiance of the N spectral bands B_i of the multispectral image. (7) Relative global dimensional synthesis error index (ERGAS) [13]. It is defined as:

2 2 1 ( ) 1 100 N i i i

RM SE B

h ERG AS

l N  M

 

 _ _

 



(15)

(4)

1(a), (b). The fused images with different fusion methods based on DWT, SWT, DTCWT, CT, NSCT and the proposed scheme are shown in Figure 1 (c)-(h).

(a) Panchromatic image (b) Multispectral image

(e) Fused image using DTCWT-based method (f) Fused image using CT-based method

(g) Fused image using NSCT-based method (h) Fused image using the proposed scheme Figure 2. Source images and the fused images (512*512 pixels)

(5)

in Table 1. The best results are indicated in bold. From Table 1, we can see that the proposed scheme takes all the best objective evaluations, which is obviously better than the other methods.

Table 1. The objective evaluations of Fig.1(c)~(h)

Methods Band MI CC PSNR UIQI RMSE RASE ERGAS

DWT

R 0.7586 0.7763 65.3668 0.4821 0.1279

49.8701 20.3074

G 0.7177 0.6942 64.3869 0.4801 0.1539

B 0.7275 0.7557 65.9968 0.4819 0.1375

SWT

R 0.9426 0.8735 68.1332 0.5367 0.1000

42.1649 14.5158

G 0.8989 0.8334 67.5575 0.5390 0.1068

B 0.9437 0.8673 68.7050 0.5410 0.0936

DTCWT

R 0.9092 0.8635 68.4625 0.4936 0.1098

42.7707 14.9359

G 0.8683 0.8210 67.3205 0.4942 0.1098

B 0.9086 0.8583 67.8679 0.5015 0.0963

R 0.7328 0.7611 65.4316 0.4821 0.1364

49.6298 20.1121

G 0.7175 0.6991 64.4754 0.4793 0.1523

B 0.7604 0.7796 66.0960 0.4836 0.1264

NSCT

R 0.9498 0.8735 67.8963 0.5441 0.1027

42.8416 14.9854

G 0.9089 0.8316 67.2530 0.5453 0.1106

B 0.9408 0.8624 68.4183 0.5412 0.0967

Proposed

R 1.1802 0.9195 70.5419 0.6649 0.0758

37.2446 11.3254

G 1.1360 0.8852 69.7726 0.6679 0.0828

B 1.1848 0.9107 70.5144 0.6659 0.0760

Table 2. The objective evaluations of Figure 2(c)~(h)

Method Band MI CC PSNR UIQI RMSE RASE ERGAS

DWT

R 0.9673 0.8918 66.6282 0.6086 0.1189

29.0186 10.0380

G 0.9630 0.8849 66.5295 0.6192 0.1202

B 0.9462 0.8849 66.3133 0.6007 0.1233

SWT

R 1.4253 0.9583 71.0369 0.6627 0.0716

22.5566 6.0651

G 1.4297 0.9566 70.9183 0.6745 0.0725

B 1.3989 0.9532 70.6476 0.6584 0.0748

DTCWT

R 1.3734 0.9544 70.7459 0.6059 0.0740

22.9493 6.2781

G 1.3785 0.9525 70.6176 0.6205 0.0751

B 1.3470 0.9484 70.3403 0.6021 0.0775

R 0.9666 0.9002 66.7134 0.5959 0.1177

28.8941 9.9521

G 0.9595 0.8946 66.6071 0.6078 0.1192

B 0.9392 0.8872 66.3755 0.5900 0.1224

NSCT

R 1.4320 0.9582 71.1138 0.6873 0.0709

22.4258 5.9950

G 1.4342 0.9567 71.0268 0.6982 0.0716

B 1.4008 0.9534 70.7642 0.6815 0.0738