Anti-interference Mechanism of ICA Blind Source Separation Combined with Wavelet De-noising

(1)

DOI:_{10.12928/TELKOMNIKA.v14i2A.4320} ₂₆₅

Anti-interference Mechanism of ICA Blind Source

Separation Combined with Wavelet De-noising

Liu Sheng*1, Zhou Shuanghong2, Li Bing3, Zhang Lanyong4

1,3,4

College of Automation, Haerbin Engineering University, Haerbin Heilongjiang, 150001 China 2_{College of Science, Haerbin Engineering University, Haerbin Heilongjiang, 150001 China}

*Corresponding author, e-mail: 37645694@qq.com Abstract

In the issues of signal de-noising, utilize K-SVD and other classical dictionary learning algorithm for sparse decomposition and reconstruction of signal, which cannot effectively eliminate influence of noise. The method suggested by this paper makes some improvement to the classical dictionary learning. Firstly, utilize K-SVD algorithm to make the dictionary learning; then, utilize the method of non-linear least squares to fit each atom in the dictionary and get the revised dictionary; finally, utilize the method of Particle Swarm Optimization to solve the spare representation of signal and get the reconstructed signal at last. It is proved through the experience that the de-noising effect of this paper is obvious superior to the conventional dictionary learning algorithm and is close to the effect of wavelet analytical approach.

Keywords: Dictionary learning; Blind-source signal; Wavelet de-nosing; Signal reconstruction

1. Introduction

The problem of signal de-noising is a classical problem in the signal processing [1]. As a classical de-noising algorithm, wavelet analytic is widely used in the de-noising problem because of its good time-frequency analytical characteristic [2, 3]. But this kind of method relies much on the priori knowledge. Before de-noising, firstly, wavelet base and decomposition level must be determined to be used according to the signal types. The existing experience proved that different wavelet base and decomposition selected to be used have great difference of signal de-noising effect to the different types of signals. So, this causes some problems to the de-noising of applying wavelet decomposition de-noising. In recent years, Empirical Mode Decomposition (EMD) [4, 5] has been widely researched as a new time-frequency analysis method. Its principle is based on the large number of data analysis to make the self-adapting signal decomposition for signal. Its advantage is that it is not necessary to give the wavelet and decomposition level in advance, but to get in accordance with the signal characteristic self-adapting. Experiences have shown that this method can require very good de-nosing effect. But it also has certain dependence on the priori message. Sparse representation has the natural advantage in signal de-noising while it does not need to any priori messages of signal and noise. And it has become the researching hot spot in recent years. Dictionary learning is the researching hot spot in field of sparse representation. Conventional dictionary learning algorithms have K-SVD and RLS-DLA. Effects of these algorithms directly used for signal de-noising is not so good. Its main reason is that the dictionary trained by utilizing the observed signals (signal with the noise) is also polluted by the noise. Thus, the Signal to Noise Ratio of reconstructed signals is not improved obviously.

2. Research Method 2.1. Problem Models

In the issues of signal de-noising, utilizing the dictionary learning method to reconstruct signal is the effective approach to noise removal/ But because of the feature that dictionary learning mechanism will make atoms of dictionary to be close to the received signals, the dictionary trained by received signals will also be influenced by noise. For this, the improved method suggested by this paper is: firstly, use K-SVD to make dictionary train for the received signals and get dictionary D. Then, through the way of curve fitting, fit each atom in the dictionary and thus get the settled dictionary ′. Finally, for the new ′, we can utilize the method

(2)

of particle swarm optimization to calculate the corresponding sparse coefficient vector w and get the reconstructed signal

x

ˆ



D w



Dictionary learning algorithm mainly solves the following problems:





, ,

arg min , arg min

. 1 ,

p D W

D W

f D W X DW

s t i L w s

 

   

(1)

Thereinto, ∈ is the trained signal, ∈ is the dictionary needed to be trained, ∈ is sparse coefficient matrix, and s is the sparse constraint. The method suggested by this paper is that extract the characteristic of received signal mainly through K-SVD method, then make it more close to the characteristic of non-noise signal through the method of curve fitting and calculate the sparse coefficient vector through the algorithm of particle swarm optimization. Because it is the efficient, fast and excellent performance possessed by modern intelligent optimization algorithm, particle swarm optimization can solve the non-linear problems and reserve the details in the signal. Thus, the algorithm suggested by this paper can efficiently solve the problem of signal de-noising.

x D w

Figure 1. Algorithm Flow Chart

2.2. Algorithm Description

Classical problem of signal de-noising is described as following: x is the signal without being polluted by noise, n is the Gauss white noise and y is the observed signal. Namely:

y

 

x n

(2)

The aim of signal de-noising is trying to remove the influence of noise and recover the original signal. It is actually needed to solve the optimization problem while making signal de-noising through dictionary learning. D is the dictionary to be learned and w is the coefficient vector.





0 2

, arg m in

opt opt

D w

D w  w  yD w (3)

Basic idea of K-SVD method is: firstly fix D to calculate w and then utilize optimized w

to calculate D. Its basic principle is calculating the residual matrix , and making SVD composition for the matrix to upgrade D.

(3)

[6-10] But the experiments have shown that the de-noising effect of this method is not ideal. It is because that algorithm of classical dictionary learning is to get the sparse representation by utilizing the observed signals to make the atoms in dictionary more close to the observed signals. So, when the observed signals are polluted by the noise, the atoms in the dictionary obtained by training also must be close to the signals with noise. For this problem, correction of atoms in the dictionary is considered to make them influence less by the noise. Scheme suggested by this paper is fitting each atom in the dictionary utilizing the method of non-linear least square.

(a) Atoms with noise (b) Fitted atoms

Figure 2. Atom Curve and Its Fitting Curve in the Dictionary

From the Figure 1 (a), it can be seen that atoms trained by K-SVD are still influenced by the noise. From the Figure 2 (b), it can be seen that atoms in the dictionary presented a smooth curve after applying the non-linear least squares fitting, thus efficiently reducing the influence of noise.

Basic principle of non-linear least square is as following: given a map _: _→ and solve the following problems:





2 2

2 1

arg min ( )

( ) 0.5 ( ( )) 0.5 ( )

opt _A m

i i

A F A

F A f A f A

       _ 



(4)

After getting the new dictionary D via curve fitting, its corresponding spare vector w

shall be calculated. So, in the case of having certain D, solving w namely is solving the following optimization problems.

0 2

arg m in

o pt _w

w  w  yD w (5)

While this optimization problem is a NP problem, and therefore, this paper uses method of parcel swarm optimization to solve w. Main idea of the algorithm is from the research of birds’ predation.

PSO (Particle Swam Optimization) is a kind of evolutionary algorithm simulating birds’ predation which can gradually be close to the optimization via the influence among each individual, individuals and overall situation in the particle swarm. PSO algorithm is flexible, has strong global searching ability and quicker convergence speed. And PSO absolutely does not need the priori messages of solution while seeking the optimization and has the innate advantage while seeking the optimized search sparse solution. In addition, while solving the non-linear problem, PSO has certain advantage [11, 12].

(4)

In PSO algorithm, positions and speeds of a group of random particles must be firstly initialized and particles are needed to continuously update itself according to the individual optimal value pbest and global optimal value gbest in the iteration. Updating formula of particles’ speed and position is

1 1

(

)

2 2

(

)

k k k k k k

i i i i i

v





mv



c rand pbest



w



c rand gbest



w

(6)

1 1

k k k

i i i

w





w



v

 (7)

Thereinto, m is inertia coefficient that is conducive to algorithm convergence with which it is descending following with the iterations times and m in this paper is initially set as 1 and descends to 0.2 following with algorithm process; and are acceleration coefficients which are used to adjust the step size of particles flying to the global optimal point and self optimal point and classical value 1.494 [13-17] is selected in this paper while the dimension of value will influence the veracity of algorithm solution; _, is the random number of [0,1].

Figure 3. Schematic for Particle Updating

Its specific algorithm step is as following:

Step1: initialization. Randomly initialize n (number of particles) particles’ position and speed , that is, initialize the current position of each particle to be pbest and work out the corresponding fitness S. Among all selected particles, the particle corresponding to the optimal fitness is global optimal point gbest.

0 2

S



w



y



Dw

(8)

Step2: evaluate each particle, calculate its fitness respectively and conduct following process: if the fitness of particle is superior to pbest, then set it as the new pbest; if the fitness of particle id superior to gbest, then set it as the new gbest.

Step3: update the position and speed of particle via formula (6) and (7).

Step4: detect whether reach the end condition; if reached, then stop the iteration and output the gbest; if not, skip to Step2.

3. Results and Discussion

3.1. Comparison between Algorithm in this Paper and Other Dictionary Learning Algorithm

Experiment content in this section manly is to compare the de-noising effect of algorithm suggested by this paper with the de-noising effect of conventional dictionary learning algorithms- K-SVD and RLS-DLA. PSO applied in this paper is selected 100 particles and has 100 times of iterations [18, 19].

(5)

Select input signal waveform is Figure 4(a); after adding the noise, Signal to Noise Ratio is 10dB and its waveform is Figure 4(b); de-noising process it respectively utilizing three kinds of algorithms and the result is as following:

(a) Signal Waveform without Noise (b) Signal Waveform after Adding Noise

(d) Reconstructed Signal of K-SVD Algorithm (e) Reconstructed Signal of RLS-DLA Algorithm Figure 4. Comparisons among Reconstructed Signals

(6)

From Point A, it can be seen that the algorithm suggested by this paper has reasonable effect in details process. The main reason is utilizing the method of particle swarm optimization to solve the sparse representation vector which is good for recovering the characteristics of original signal.

This paper also makes comparison analysis and 220 times of Monte Carlo experiments among the de-noising effect of three kinds of algorithms under different Signal to Noise Ratio and the results are shown as Table 1. From these data comparisons, it can be known that de-noising effect of the algorithm suggested by this paper is obvious superior to K-SCD and RLS-DLA. Meantime, it also can be seen that the effect difference between K-SVD algorithm and algorithm suggested by this paper gradually decreases while the input Signal to Noise Ratio gradually increases.

Table 1. De-noising Effect of Algorithm under Different Signal to Noise Ratio

Method SNR/MSE

10dB 15dB 20dB 25dB Algorithm Suggested by

This paper 23.076/0.0023 25.272/0.0018 27.581/0.0012 30.321/0.0010

K-SVD 14.121/0.0203 17.463/0.0166 22.485/0.0025 27.627/0.0012

RLS-DLA 10.713/0.0310 15.609/0.0157 20.491/0.0049 25.434/0.0017

From the experiment results above, it can be very clear seen that the method suggested by this paper has the best de-noising effect among three kinds of dictionary learning algorithms.

3.2. Apply the Algorithm by this Paper in Four Kinds of Typical Signals

In addition, to prove the extended use of this algorithm, four kinds of typical signals- “Blocks”, “Bumps”, “Heavy sine” and “Doppler” are selected to make signal de-noising experiment. The result is as following:

Amp

litud

Amplitude

(7)

Figure 6. De-noising Effect of “Bumps” Signal

Figure 7. De-noising Effect of “Heavy sine” Signal

(8)

In experiments above, Signal to Noise Ratio is selected to be 15dB. From the result, it can be seen that the algorithm suggested by this paper can be better applied in various signals.

At last, this paper selects typical signal “Doppler” as the sample to make the comparison of de-noising effects between the algorithm suggested by this paper and classical de-noising algorithm-Bayesian Wavelet. Thereinto, wavelet base of Bayesian Wavelet is selected “db8” wavelet, decomposition level is 10 and hard threshold value method is applied. Make 200 times of Monte Carlo experiments and respectively calculate the results of two kinds of algorithms-SNR and MSE. Experiment results are shown as Figure 9 and Figure 10. It can be seen that the effect of algorithm suggested by this paper is close to that of Bayesian Wavelet.

Figure 9. Change of SNR Which Has Removed Following with Input SNR

Figure 10. Change of MSE Which Following with Input MSE

4. Conclusions

Sparse representation and dictionary learning methods have many innate advantages in de-noising. Thus, research on this direction has relatively important sense. This paper has suggested a kind of signal de-noising algorithm based on dictionary learning and particle swarm optimization. This algorithm makes itself more suitable for signal de-noising problem and makes de-noising effect more obvious though improving K-SVD and adding particle swarm algorithm. Experiments have shown that the method suggested by this paper has better de-noising effect than conventional dictionary learning K-SVD and RLS-DLA.

(9)

References

[1] Jinyu Hu, Zhiwei Gao. Distinction immune genes of hepatitis-induced heptatocellular carcinoma.

Bioinformatics. 2012; 28(24): 3191-3194.

[2] Cui Y, Zhang S, Chen Z, Zheng W. A New Digital Image Hiding Algorithm Based on Wavelet Packet Transform and Singular Value Decomposition. TELKOMNIKA Indonesian Journal of Electrical

Engineering. 2014; 12(7): 5408-5413.

[3] Lv Z, Li X, Zhang B, Wang W, Zhu Y, Hu J, Feng S. Managing Big City Information Based on WebVRGIS. IEEE Access. 2016; 4: 407-415.

[4] M Abdar, SRN Kalhori, T Sutikno, IMI Subroto, G Arji. Comparing Performance of Data Mining Algorithms in Prediction Heart Diseases. International Journal of Electrical and Computer

Engineering (IJECE). 2015; 5(6): 1569-1576.

[5] Yang J, Lin Y, Gao Z. Quality Index for Stereoscopic Images by Separately Evaluating Adding and Subtracting. PloS one. 2015; 10(12): e0145800.

[6] Jiang D, Xu Z, Lv Z. A multicast delivery approach with minimum energy consumption for wireless multi-hop networks. Telecommunication Systems. 2015: 1-12.

[7] Liu Y, Yang J, Meng Q. Stereoscopic image quality assessment method based on binocular combination saliency model. Signal Processing. 2016; 125: 237-248.

[8] Lv Z, Yin T, Han Y. WebVR-web virtual reality engine based on P2P network. Journal of Networks. 2011; 6(7): 990-998.

[9] Yang J, Zhou J, Lv Z. A Real-Time Monitoring System of Industry Carbon Monoxide Based on Wireless Sensor Networks. Sensors. 2015; 15(11): 29535-29546.

[10] Jiang D, Ying X, Han Y. Collaborative multi-hop routing in cognitive wireless networks. Wireless

Personal Communications. 2016; 86(2): 901-923.

[11] Shereen H. Ali, Ali I. El Desouky, Ahmed I. Saleh. A New Profile Learning Model for Recommendation System based on Machine Learning Technique. Indonesian Journal of Electrical

Engineering and Informatics. 2016; 4(1): 81-92.

[12] Hui Z. The Design of Electronic Toll Collection System Based on Radio-Frequency Identification.

Bulletin of Electrical Engineering and Informatics. 2013; 2(4): 286-292.

[13] Jinyu Hu and Zhiwei Gao. Distinction immune genes of hepatitis-induced heptatocellular carcinoma.

Bioinformatics. 2012; 28(24): 3191-3194.

[14] Aluvalu Rajani Kanth, Vanraj Kamliya, Lakshmi Muddana. HASBE Access Control Model with Secure Key Distribution and Efficient Domain Hierarchy for Cloud Computing. International Journal of

Electrical and Computer Engineering (IJECE). 2016; 6(2): 770-777.

[15] Saxena S. Extension to HiRLoc Algorithm for Localization Error Computation in Wireless Sensor Networks. Indonesian Journal of Electrical Engineering and Informatics (IJEEI). 2013; 1(4): 119-126. [16] Geng Y, Chen J, Fu R, Bao G, Pahlavan K. Enlighten wearable physiological monitoring systems:

On-body rf characteristics based human motion classification using a support vector machine. IEEE

Transactions on Mobile Computing, 2016; 15(3), 656-671.

[17] Song X, Geng Y. Distributed Community Detection Optimization Algorithm for Complex Networks.

Journal of Networks. 2014; 9(10): 2758-2765.

[18] Pahlavan K, Krishnamurthy P, Geng Y. Localization Challenges for the Emergence of the Smart World. Access, IEEE. 2015; 3(1): 1-11.

[19] He J, Geng Y, Wan Y, Li S, Pahlavan K. A cyber physical test-bed for virtualization of RF access environment for body sensor network. Sensors Journal, IEEE. 2013; 13(10): 3826-3836.

(1)

1 1

(

)

2 2

(

)

k k k k k k

i i i i i

v





mv



c rand pbest



w



c rand gbest



w

(6)

1 1

k k k

i i i

w





w



v

 (7)

Figure 3. Schematic for Particle Updating

Its specific algorithm step is as following:

0 2

S



w



y



Dw

(8)

Step3: update the position and speed of particle via formula (6) and (7).

Step4: detect whether reach the end condition; if reached, then stop the iteration and output the gbest; if not, skip to Step2.

3. Results and Discussion

3.1. Comparison between Algorithm in this Paper and Other Dictionary Learning Algorithm

(2)

(a) Signal Waveform without Noise (b) Signal Waveform after Adding Noise

(d) Reconstructed Signal of K-SVD Algorithm (e) Reconstructed Signal of RLS-DLA Algorithm Figure 4. Comparisons among Reconstructed Signals

(3)

Table 1. De-noising Effect of Algorithm under Different Signal to Noise Ratio

Method SNR/MSE

10dB 15dB 20dB 25dB Algorithm Suggested by

This paper 23.076/0.0023 25.272/0.0018 27.581/0.0012 30.321/0.0010 K-SVD 14.121/0.0203 17.463/0.0166 22.485/0.0025 27.627/0.0012 RLS-DLA 10.713/0.0310 15.609/0.0157 20.491/0.0049 25.434/0.0017

From the experiment results above, it can be very clear seen that the method suggested by this paper has the best de-noising effect among three kinds of dictionary learning algorithms.

3.2. Apply the Algorithm by this Paper in Four Kinds of Typical Signals

Amp

litud

Amplitude

(4)

Figure 6. De-noising Effect of “Bumps” Signal

Figure 7. De-noising Effect of “Heavy sine” Signal

(5)

In experiments above, Signal to Noise Ratio is selected to be 15dB. From the result, it can be seen that the algorithm suggested by this paper can be better applied in various signals.

Figure 9. Change of SNR Which Has Removed Following with Input SNR

Figure 10. Change of MSE Which Following with Input MSE

4. Conclusions

(6)

References

[1] Jinyu Hu, Zhiwei Gao. Distinction immune genes of hepatitis-induced heptatocellular carcinoma. Bioinformatics. 2012; 28(24): 3191-3194.

[2] Cui Y, Zhang S, Chen Z, Zheng W. A New Digital Image Hiding Algorithm Based on Wavelet Packet Transform and Singular Value Decomposition. TELKOMNIKA Indonesian Journal of Electrical Engineering. 2014; 12(7): 5408-5413.

[3] Lv Z, Li X, Zhang B, Wang W, Zhu Y, Hu J, Feng S. Managing Big City Information Based on WebVRGIS. IEEE Access. 2016; 4: 407-415.

[4] M Abdar, SRN Kalhori, T Sutikno, IMI Subroto, G Arji. Comparing Performance of Data Mining Algorithms in Prediction Heart Diseases. International Journal of Electrical and Computer Engineering (IJECE). 2015; 5(6): 1569-1576.

[5] Yang J, Lin Y, Gao Z. Quality Index for Stereoscopic Images by Separately Evaluating Adding and Subtracting. PloS one. 2015; 10(12): e0145800.

[6] Jiang D, Xu Z, Lv Z. A multicast delivery approach with minimum energy consumption for wireless multi-hop networks. Telecommunication Systems. 2015: 1-12.

[7] Liu Y, Yang J, Meng Q. Stereoscopic image quality assessment method based on binocular combination saliency model. Signal Processing. 2016; 125: 237-248.

[8] Lv Z, Yin T, Han Y. WebVR-web virtual reality engine based on P2P network. Journal of Networks. 2011; 6(7): 990-998.

[9] Yang J, Zhou J, Lv Z. A Real-Time Monitoring System of Industry Carbon Monoxide Based on Wireless Sensor Networks. Sensors. 2015; 15(11): 29535-29546.

[10] Jiang D, Ying X, Han Y. Collaborative multi-hop routing in cognitive wireless networks. Wireless Personal Communications. 2016; 86(2): 901-923.

[11] Shereen H. Ali, Ali I. El Desouky, Ahmed I. Saleh. A New Profile Learning Model for Recommendation System based on Machine Learning Technique. Indonesian Journal of Electrical Engineering and Informatics. 2016; 4(1): 81-92.

[12] Hui Z. The Design of Electronic Toll Collection System Based on Radio-Frequency Identification. Bulletin of Electrical Engineering and Informatics. 2013; 2(4): 286-292.

[13] Jinyu Hu and Zhiwei Gao. Distinction immune genes of hepatitis-induced heptatocellular carcinoma. Bioinformatics. 2012; 28(24): 3191-3194.

[14] Aluvalu Rajani Kanth, Vanraj Kamliya, Lakshmi Muddana. HASBE Access Control Model with Secure Key Distribution and Efficient Domain Hierarchy for Cloud Computing. International Journal of Electrical and Computer Engineering (IJECE). 2016; 6(2): 770-777.

[15] Saxena S. Extension to HiRLoc Algorithm for Localization Error Computation in Wireless Sensor Networks. Indonesian Journal of Electrical Engineering and Informatics (IJEEI). 2013; 1(4): 119-126. [16] Geng Y, Chen J, Fu R, Bao G, Pahlavan K. Enlighten wearable physiological monitoring systems:

On-body rf characteristics based human motion classification using a support vector machine. IEEE Transactions on Mobile Computing, 2016; 15(3), 656-671.

[17] Song X, Geng Y. Distributed Community Detection Optimization Algorithm for Complex Networks. Journal of Networks. 2014; 9(10): 2758-2765.

[18] Pahlavan K, Krishnamurthy P, Geng Y. Localization Challenges for the Emergence of the Smart World. Access, IEEE. 2015; 3(1): 1-11.

[19] He J, Geng Y, Wan Y, Li S, Pahlavan K. A cyber physical test-bed for virtualization of RF access environment for body sensor network. Sensors Journal, IEEE. 2013; 13(10): 3826-3836.

Anti-interference Mechanism of ICA Blind Source Separation Combined with Wavelet De-noising