Multisensor Fusion and Integration in the Wake of Big Data, Deep Learning and Cyber Physical System

  Sukhan Lee · Hanseok Ko Songhwai Oh    Editors Multisensor Fusion and Integration in the

Wake of Big Data, Deep

Learning and Cyber Physical System

  An Edition of the Selected Papers from

the 2017 IEEE International Conference

on Multisensor Fusion and Integration for Intelligent Systems (MFI 2017)

  

Lecture Notes in Electrical Engineering

Volume 501

  Board of Series editors Leopoldo Angrisani, Napoli, Italy Marco Arteaga, Coyoacán, México Bijaya Ketan Panigrahi, New Delhi, India Samarjit Chakraborty, München, Germany Jiming Chen, Hangzhou, P.R. China Shanben Chen, Shanghai, China Tan Kay Chen, Singapore, Singapore Rüdiger Dillmann, Karlsruhe, Germany Haibin Duan, Beijing, China Gianluigi Ferrari, Parma, Italy Manuel Ferre, Madrid, Spain Sandra Hirche, München, Germany Faryar Jabbari, Irvine, USA Limin Jia, Beijing, China Janusz Kacprzyk, Warsaw, Poland Alaa Khamis, New Cairo City, Egypt Torsten Kroeger, Stanford, USA Qilian Liang, Arlington, USA Tan Cher Ming, Singapore, Singapore Wolfgang Minker, Ulm, Germany Pradeep Misra, Dayton, USA Sebastian Möller, Berlin, Germany Subhas Mukhopadyay, Palmerston North, New Zealand Cun-Zheng Ning, Tempe, USA Toyoaki Nishida, Kyoto, Japan Federica Pascucci, Roma, Italy Yong Qin, Beijing, China Gan Woon Seng, Singapore, Singapore Germano Veiga, Porto, Portugal Haitao Wu, Beijing, China Junjie James Zhang, Charlotte, USA

• ** Indexing: The books of this series are submitted to ISI Proceedings, EI-Compendex,

  SCOPUS, MetaPress, Springerlink **

  Lecture Notes in Electrical Engineering (LNEE) is a book series which reports the latest research

  and developments in Electrical Engineering, namely: Communication, Networks, and Information Theory
• Computer Engineering
• Signal, Image, Speech and Information Processing
• Circuits and Systems
• Bioengineering
• Engineering

• The audience for the books in LNEE consists of advanced level students, researchers, and industry

  professionals working at the forefront of their fields. Much like Springer’s other Lecture Notes

  series, LNEE will be distributed through Springer’s print and electronic publishing channels.

  

For general information about this series, comments or suggestions, please use the contact

address under “service for this series”.

To submit a proposal or request further information, please contact the appropriate Springer

Publishing Editors:

  Asia: China, Jessie Guo, Assistant Editor (jessie.guo@springer.com) (Engineering) India, Swati Meherishi, Senior Editor (swati.meherishi@springer.com) (Engineering) Japan, Takeyuki Yonezawa, Editorial Director (takeyuki.yonezawa@springer.com) (Physical Sciences & Engineering) South Korea, Smith (Ahram) Chae, Associate Editor (smith.chae@springer.com) (Physical Sciences & Engineering) Southeast Asia, Ramesh Premnath, Editor (ramesh.premnath@springer.com) (Electrical Engineering) South Asia, Aninda Bose, Editor (aninda.bose@springer.com) (Electrical Engineering) Europe: Leontina Di Cecco, Editor (Leontina.dicecco@springer.com)

(Applied Sciences and Engineering; Bio-Inspired Robotics, Medical Robotics, Bioengineering;

Computational Methods & Models in Science, Medicine and Technology; Soft Computing;

Philosophy of Modern Science and Technologies; Mechanical Engineering; Ocean and Naval

Engineering; Water Management & Technology) (christoph.baumann@springer.com)

(Heat and Mass Transfer, Signal Processing and Telecommunications, and Solid and Fluid

Mechanics, and Engineering Materials) North America: Michael Luby, Editor (michael.luby@springer.com) (Mechanics; Materials) More information about this series at

• ^• Sukhan Lee Hanseok Ko Songhwai Oh

  Editors Multisensor Fusion and Integration in the Wake of Big Data, Deep Learning and Cyber Physical System An Edition of the Selected Papers

from the 2017 IEEE International Conference

on Multisensor Fusion and Integration for Intelligent Systems (MFI 2017)

  123 Editors Sukhan Lee Songhwai Oh Intelligent Systems Research Institute Department of Electrical and Computer Sungkyunkwan University Engineering Suwon Seoul National University Korea (Republic of) Seoul

  Korea (Republic of) Hanseok Ko School of Electrical Engineering Korea University Seoul Korea (Republic of)

ISSN 1876-1100

  ISSN 1876-1119 (electronic) Lecture Notes in Electrical Engineering

ISBN 978-3-319-90508-2

  ISBN 978-3-319-90509-9 (eBook) https://doi.org/10.1007/978-3-319-90509-9 Library of Congress Control Number: 2018940915 © Springer International Publishing AG, part of Springer Nature 2018

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part

of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,

recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission

or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this

publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this

book are believed to be true and accurate at the date of publication. Neither the publisher nor the

authors or the editors give a warranty, express or implied, with respect to the material contained herein or

for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by the registered company Springer International Publishing AG part of Springer Nature The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Preface

  Multisensor fusion and integration is playing a critical role in harnessing the smart technologies as we ride the big wave of the 4th Industrial Revolution. Deployment of the Internet of Things, Cyber-Physical Systems and Robotics in distributed environment is rapidly rising as our society seeks to transition from being ambient to being smart and, at the same time, to enable human to curate information and knowledge between ubiquitous and collective computing environments. What surround us are the networks of sensors and actuators that monitor our environment, health, security and safety, as well as the service robots, intelligent vehicles and autonomous systems of ever heightened autonomy and dependability with inte- grated heterogeneous sensors and actuators. Developing fundamental theories and advancing implementation tools to address the emerging key issues in multisensor fusion and integration in the wake of big data and deep learning would make the above transition smooth and rewarding.

  This volume is an edition of the papers selected from the 13th IEEE International Conference on Multisensor Integration and Fusion, IEEE MFI 2017, held in Daegu, Korea, 16–22 November 2017. Only 17 papers out of the 112 papers accepted for

  IEEE MFI 2017 were chosen and requested for revision and extension to be included in this volume. The 17 contributions to this volume are organized into two chapters: Chapter 1 is dedicated to the theories in data and information fusion in distributed environment and Chapter 2 to the multisensor fusion in robotics. To help readers understand better, a chapter summary is included in each chapter as an introduction.

  It is the wish of the editors that readers find this volume informative and enjoyable. We would also like to thank Springer-Verlag for undertaking the pub- lication of this volume.

  Sukhan Lee Hanseok Ko

  Songhwai Oh Contents

  Multi-sensor Fusion: Theory and Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

   Sukhan Lee and Muhammad Abu Bakr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

   Florian Rosenthal, Benjamin Noack, and Uwe D. Hanebeck . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

   Qiang Liu and Nageswara S. V. Rao . . . . . . . . . . . . . . . . . . . . . . . . . . .

   Justin D. Brody, Anna M. R. Dixon, Daniel Donavanik, Ryan M. Robinson, and William D. Nothwang . . . . . . . . . . . . . . . . . . . . . . .

   Camila Ramirez and Nageswara S. V. Rao . . .

   Tran Tuan Nguyen, Jens Spehr, Jonas Sitzmann, Marcus Baum, Sebastian Zug, and Rudolf Kruse . . . . . . . . . . . . . . . . . . Achim Kuwertz, Dirk Mühlenberg, Jennifer Sander, and Wilmuth Müller

  . . . . . . . . . . .

   Gaochao Feng, Deqiang Han, Yi Yang, and Jiankun Ding viii Contents

  Multi-sensor Fusion Applications in Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

   Sangwook Park, Chul Jin Cho, Younglo Lee, Andrew Da Costa, SangHo Lee, and Hanseok Ko . . . . . . . . . . . . . . . . . .

   Daniel Bender, Wolfgang Koch, and Daniel Cremers Mårten Lager, Elin A. Topp, and Jacek Malec Mohammad Aldibaja, Noaki Suganuma, Keisuke Yoneda, Ryo Yanase, and Akisue Kuramoto Johannes Buyer, Martin Vollert, Mihai Kocsis, Nico Sußmann, and Raoul Zöllner Fabian Sigges and Marcus Baum . . . . . . . . . . . .

   Xiuyi Fan, Huiguo Zhang, Cyril Leung, and Zhiqi Shen Yanyan Bao, Fuchun Sun, Xinfeng Hua, Bin Wang, and Jianqin Yin Qingcong Wu and Ziyan Shao

  

Multi-sensor Fusion:

Theory and Practice

  

Multi-sensor Fusion: Theory and Practice

  Sukhan Lee and Hanseok Ko Multisensor fusion and integration in a distributed environment is becoming of utmost importance, especially, in the wake of the growing deployment of Internet of Things (IoT) as well as Cyber Physical Systems (CPS). Although the fundamental theory behind multisensor fusion and integration has been well-established through several decades of investigations, in practice, there still remain a number of technical chal- lenges to overcome, in particular, for dealing with multisensor fusion and integration in a distributed environment. Specifically speaking, multisensor fusion with the known cross-correlations among multiple data sources can be handled ideally, for instance, by Bar-Shalom Campo and Generalized Millman’s formula. However, in a distributed environment, a number of critical issues arise that are yet to be addressed and solved, including (1) the difficulty of estimating exact cross-correlations among multiple data sources due to the physical relationships possibly existing among their observations as well as the possible double counting by sharing prior information or data sources, (2) the presence of inconsistency or outliers among data sources, (3) the existence of transmission delays as well as data losses and (4) the incorporation of various con- straints that may be available among states and observations into fusion. The papers collected for this chapter are to address some of the critical issues as described above in a theoretical and/or a practical point of view, as follows:

  The paper, entitled “Covariance Projection as General Framework of Data Fusion and Outlier Removal,” by Sukhan Lee and Muhammad Abu Bakr proposes a general framework of distributed data fusion for distributed sensor networks of arbitrary redundancies, where inconsistent data are identified simultaneously within the frame- work. The paper, entitled “State Estimation in Networked Control Systems with Delayed and Lossy Acknowledgments,” by Florian Rosenthal, Benjamin Noack and Uwe D. Hanebeck deals with the state estimation in networked control systems where the control inputs and measurements transmitted via networks as well as the acknowledgements packets sent by the actuator upon reception of control inputs are subject to data losses and random transmission delays. The paper, entitled “Perfor- mance of State Estimation and Fusion with Elliptical Motion Constraints,” by Qiang Liu and Nageswara Rao investigates target tracking in the presence of elliptical nonlinear constraints on its motion dynamics, where the state estimates generated by sensors are considered to be sent over long-haul lossy links to a remote fusion center. The paper, entitled “Relevance and Redundancy as Selection Techniques for Human-Autonomy Sensor Fusion,” by Justin David Brody, Anna Marie Rogers Dixon, Daniel Donavanik, Ryan M. Robinson and William D. Nothwang addresses the problem of sensor fusion in a human-autonomy system where the dynamic nature of sensors makes it difficult to model their variability. The paper examines the application of information theoretic entities, such as the relevance between sensors and target classes and the redundancy among the selected sensors, as the criteria for evaluating the Operational State Using SPRT Methods with Radiation Sensor Networks,” by Nageswara Rao and Camila Ramirez deals with the problem of inferring the opera- tional status of a reactor facility using measurements from a radiation sensor network where sensor measurements are inherently random with the parameters determined by the intensity at the sensor locations. The paper, entitled “Applying Knowledge-Based Reasoning for Information Fusion in Intelligence, Surveillance, and Reconnaissance,” by Wilmuth Muller, Achim Kuwertz, Dirk Muhlenberg and Jennifer Sander presents a method of high-level data fusion combining probabilistic information processing with logical and probabilistic reasoning. This is to support human operators in their situa- tional awareness for improving their capabilities of making efficient and effective decisions. The paper, entitled “Multiple Classifier Fusion Based on Testing Sample Pairs,” by Gaochao Feng, Deqiang Han, Yi Yang, and Jiankun Ding presents a multiple classifier system operated under the classification based on testing sample pairs, where fuzzy evidential reasoning is used to implement multiclass classification fusion. The paper, entitled “Improving Ego-Lane Detection by Incorporating Source Reliability,” by Tran Tuan Nguyen, Jens Spehr, Jian Xiong, Marcus Baum, Sebastian Zug and Rudolf Kruse proposes an efficient and sensor-independent metric which provides an objective and intuitive self-assessment for the entire road estimation process at multiple levels, including individual detectors, lane estimation and the target applications.

  

Covariance Projection as a General

Framework of Data Fusion

and Outlier Removal

_(&)

  Sukhan Lee and Muhammad Abu Bakr

  

Intelligent Systems Research Institute, Sungkyunkwan University,

Gyeonggi-do, Suwon 440-746, South Korea

{lsh1,abubakr}@skku.edu

Abstract. A fundamental issue in sensor fusion is to detect and remove outliers

as sensors often produce inconsistent measurements that are difficult to predict

and model. The detection and removal of spurious data is paramount to the

quality of sensor fusion by avoiding their inclusion in the fusion pool. In this

paper, a general framework of data fusion is presented for distributed sensor

networks of arbitrary redundancies, where inconsistent data are identified

simultaneously within the framework. By the general framework, we mean that it

is able to fuse multiple correlated data sources and incorporate linear constraints

directly, while detecting and removing outliers without any prior information.

The proposed method, referred to here as Covariance Projection (CP) Method,

aggregates all the state vectors into a single vector in an extended space. The

method then projects the mean and covariance of the aggregated state vectors

onto the constraint manifold representing the constraints among state vectors that

must be satisfied, including the equality constraint. Based on the distance from

the manifold, the proposed method identifies the relative disparity among data

sources and assigns confidence measures. The method provides an unbiased and

optimal solution in the sense of Minimum Mean Square Error (MMSE) for

distributed fusion architectures and is able to deal with correlations and uncer-

tainties among local estimates and/or sensor observations across time. Simulation

results are provided to show the effectiveness of the proposed method in iden-

tification and removal of inconsistency in distributed sensors system.

  Keywords: Covariance projection method Constraint manifold Data fusion Distributed sensor network Inconsistent data

1 Introduction

  Multisensor data fusion is to obtain a more meaningful and precise estimate of a state by combining data from multiple sources. One of the inherent issues in multisensor data fusion is that of uncertainty in sensor measurements. The sensor uncertainties may come from impreciseness and noise in the measurements, as well as, from ambiguities and inconsistencies present in the environment. The fusion methodologies should be able to model such uncertainties and combine data to provide a consistent and accurate

  Recently, distributed data fusion [

  

  

  ] is widely explored in diverse fields of engineering and control due to its superior performance over the centralized fusion in terms of flexibility, robustness to failure and cost-effectiveness in infrastructure and communication. However, the distributed architecture needs to address statistical dependency among the local estimates received from multiple nodes for fusion. This is due to the fact that local state estimates at individual nodes can be subject to same process noise [

  and to double counting, i.e., sharing same data sources among them

  [

  

Ignoring such statistical dependency or cross-correlation among multiple nodes

  leads to inconsistent results, causing divergence in data fusion [

  The fusion methodologies assume that the sensor measurements are affected by Gaussian noise only and thus the covariance of the estimate provides a good approximation of all the disturbances affecting the sensor measurements. However, in real applications, the sensor measurements may not only be affected by noise but also from unexpected situations such as short duration spike faults, sensor glitch, permanent failure or slowly developing failure due to sensor elements [

  Since these types of

  uncertainties are not attributable to the inherent noise, they are difficult to model. Due to these uncertainties, the estimates provided by sensor nodes in a distributed network may be spurious and inconsistent. Fusing these imprecise estimates with correct esti- mates can lead to severely inaccurate results

   ]. Hence, a data validation scheme is required to identify and eliminate the outliers from the fusion pool.

  Detection of inconsistency needs either a priori information often in the form of

   ]

  specific failure model(s) or data redundancy [ uses the generated residuals between the model outputs and actual measurements to detect and remove faults. For instance in [

  Nadaraya-Watson estimator and a priori

  observations are used to validate sensor measurements. Similarly, a priori system model

   ]. Researchers

  information as a reference is used to detect failures in filtered estimates

   ] based approaches for sensor

  validation. However, model-based methods either require an explicit mathematical model or need tuning and training for data validation. This restricts the usage of these methods in the case where prior information is not available or unmodeled failure occurs. A method to detect spurious data based on Bayesian framework is proposed in [

  

The method adds a term to the Bayesian formulation which has the effect of

  increasing the posterior distribution when measurement from one of the sensor is inconsistent with respect to the other. However, the method assumes independence of the sensor estimates in its analysis and may lead to incorrect rejection of true estimates or incorrect retaining of false estimates.

  This paper presents a general data fusion framework, referred to as Covariance Projection (CP) Method, to find an optimal and consistent fusion solution for multiple correlated data sources. The proposed method provides a framework for identifying and removing outliers in a distributed sensor network where only the sensor estimates may be available at the fusion center.

  1.1 Problem Statement In a distributed architecture

   ], the sensors are often equipped with a tracking covariance. Assume that each local system predicts the underlying states using fol- lowing equation, x ¼ A x þ Bu þ w _{k k 1 k 1 k 1} where A is the system matrix, B input matrix, u is the state _{k 1 input vector and ^x k 1} vector. The system process is affected by zero mean Gaussian noise w _k ^{1 with} covariance matrix Q. The sensor measurements are approximated as, z x _{k i ¼ H i k þ v k i þ e k i ; i}

  ¼ 1; . . .; n where v is Gaussian noise with covariance matrices R _{k i i ; i} ¼ 1; 2; . . .; n. The sensor measurements are also affected by unmodeled faults e . The state prediction of each _{k i} local system is updated by its own sensor measurement to compute local state estimates

  ). The local estimates are then communicated among sensor nodes or sent to a as (^x k ; P k central node for obtaining a global estimate. However, the local estimates may be correlated due to common process noise [

   ]. Furthermore, the

  estimates provided by local systems may be spurious and inconsistent due to the unmodeled sensor faults. As stated in the introduction, the majority of work needs a priori information in the form of particular failure model(s) to detect sensor faults

  While in a distributed architecture, the fusion node may have access to the

  estimated mean and covariance of the data sources only. Moreover, the cross- correlation among data sources is overlooked in traditional data validation schemes and outliers removal is mostly based on heuristics [

  This paper presents a general framework to validate and fuse correlated and uncertain data from multiple sources without any prior information. The proposed method assigns confidence measure to multiple data sources based on the distance from the constraint manifold. The method then statistically removes the inconsistent sensor estimates of arbitrary dimensions and correlations.

2 Proposed Approach

  , of the true state x, with covariances P , P and Consider unbiased estimates ^x ^{1 and ^x 2 1 2} cross-covariance matrix P . The statistical distribution, that is, the mean and covari- _{12 N} is aggregated such that it is transformed to an ance from individual sensors in R _2N along with the equality constraint between the two data sources, extended space of R that is, _{1 P 1 P 12}

  ^x ; P ¼ ; ^{1 2 ð1Þ}

  ^x ¼ T ^x ¼ ^x _{2 P P 2} ^x ¹² Figure

  shows the extended space representation as a 2D ellipsoid of two individual

  1D Gaussian estimates along with the constraint manifold. The constraint manifold is a manifestation of the relationship between the data from two sensors. The subspace of the _T

  

Fig. 1. (a) Extended space representation of two data sources and constraint manifold

(b) Whitening transform and projection, as a generalization of covariance extension method _{1 T} ].

  =2

  E , where D and E is the respective transformation that can be defined as, W ¼ D eigenvalue and eigenvector matrix of P. Applying Whitening transform, we get, _{W W T W}

  P M ^x ¼ WPW ¼ I; ¼ WM

  ¼ W^x; Figure

  shows the transformation of the ellipsoid into a unit circle after W. The _W

  mean and covariance are then projected on the constraint manifold M to get a fused result in the transformed space as shown in Fig.

  Inverse Whitening Transform is

  applied to obtain the optimal fused mean and covariance in the original space as, ₁ P ~x ¼ W r W^x ð2Þ _{1 T T}

  ~ P ¼ W P r P W ð3Þ _{r W W W W T T} where P r is the projection matrix computed as P r ¼ M M M M . It should be noted that the framework of CP method can incorporate any linear constraints among data sources without any additional processing. Using definition of various components in (

  

  

  ), we get the closed-form simplification of fused mean and covariance for CP method as, _{T 1 T 1 1} P M M P ð4Þ

  ~x ¼ M ^x _{T 1 1} ~

  P ¼ M P M ð5Þ

   . Using the values of

  The details of the simplification are provided in Appendix M

  we get the CP fused mean and covariance of two

  ; ^x; and P from ( sensor estimates as, ₁ ð P Þ P ð þ P P P Þ

  ~x ¼ P ^{2 21 1 2 12 21 ^x 1} ₁ ð6Þ þ P ð ^{1 P 21 Þ P ð 1 þ P 2 P 12 P 21 Þ 2} ₁ ^x

  ~ P ¼ P ^{1 ð P 1 P 12 Þ P ð 1 þ P 2 P 12 P 21 Þ ð P 1 P 21 Þ ð7Þ} _N

  Given n sensor estimates _{1 ; P 1 2 ; P 2 n ; P n} with ð^x Þ, ð^x Þ; . . .; ð^x Þ of a true state x 2 R known cross-covariance P _{ij ; i; j} can be used to provide the

  ¼ 1; . . .; n, _T optimal fused mean and covariance with M . Where I is the ¼ I ½ N ^{1 ; I N 2 Nn Š N}

  ; . . .; I identity matrix and N the dimension of individual data source. The proposed CP method provides an unbiased and optimal fused solution in the sense of Minimum Mean Square Error (MMSE) for a multisensor system of arbitrary redundancies.

  is an unbiased

  Theorem 1: The fused estimate ~x given by the CP method in Eq. ( ð Þ ¼ E x ð Þ: estimator of x, that is, E ~x

  Proof: Using

   ) we can write

₁

  x P r W x ð Þ ~x ¼ W ^x

  Taking expectation on both sides, we get ₁ E x P W ð ~x Þ ¼ W r E x ð ^x Þ

  E x ð ~x Þ ¼ 0

  E x ð Þ

  ð Þ ¼ E ~x where the assumption of unbiasedness is used for E x ð Þ. This conclude that the

  ð Þ ¼ E ^x fused state estimate ~x is an unbiased estimate of x. Theorem 2: The fused covariance ~ P of the CP method is smaller than the individual covariances, that is, ~ P P i ; i

  ¼ 1; 2; . . .; n: Proof: From Eq. (

  we can write

_T

_{1 1}

  ~ P M P M

  ¼ By Schwartz matrix inequality, we have _{1 2 T T T 1 2 T 2 1 2 1 1 2 1 2 1}

  ~ P P M P M P M P M P M P M

  ¼ i _{2 1 T 2 1} ⁱ P M P M ¼ P _{i i i}

  T

  where M is the constraint among data sources and M i ¼ I ½ Ni Š is the con- ; 0; . . .; 0 straint matrix for P i . The equality holds for P i ¼ P ij ; that is, ~ P ¼ P i ; when P i ¼ P ij ; j ¼ 1; 2; . . .; n.

  Since the estimates of the state provided by sensors in a distributed architecture are correlated, computation of cross-covariance P ; is needed to compute the fused mean _ij (

  The cross-covariance between the sensor estimates can be

  computed as [

   _{h i k 1 T T T}

  P ij ¼ I ½ K i H i Š AP A þ BQB _ij

  I K j H j ð8Þ where K and K are the Kalman gain of sensor i and j respectively for i _{i j} ; j _{k 1}

  ¼ 1; . . .; n and P represent the cross covariance of the previous cycle between sensor i and j. _ij

3 Confidence Measure of Data Sources

  The working of fusion algorithms is based on assumption that the input sensor esti- mates are consistent and consequently fails in the case of inconsistent estimates. Hence, a data validation scheme is required to identify and eliminate the outliers before fusion. The proposed approach identifies relative disparity and confidence measure of the multi-sensory data by utilizing the relationship among data sources. Assuming that the data sources can be represented jointly as a multivariate normal distribution, the confidence of data sources can be measured by calculating the distance from the constraint manifold as depicted in Fig. _N . Suppose that we have n Gaussian data with corresponding joint mean and covariance matrices as, sources in R _{2 3 2 3 N 1} P P _{1 12 . . . P 1n}

  ^x _{T N 2 P P 6} . _{7 6} ^x _{7 . . . 6} ^{12 2} _{7 6 7} ..

  P ^x ¼ . ; ¼ . _{6 . 7 4 5 . 4} . ₅ .. . .

  .. .. _T .. ^x Nn P . . . _{1n P} . . . _n

  Then the distance d from the manifold representing confidence measure can be com- puted as, _{T 1} d ð Þ P ð Þ ð9Þ ¼ ^x ~x ^x ~x

   ). For instance,

  where ~x is the point on the manifold and can be obtained by using _{1 2 and respective covariance matrices N} given two data sources with mean as ^x , ^x _{N N}

2 R

  P and P . The distance d can be obtained as, _{1 2}

  2 R _{1 T T ^x ~x} P ^{1 1} d ¼ ð ^{1 Þ ð 2 Þ}

  ^x ~x ^x ~x P ^{2 2}

  ^x ~x

  

Fig. 2. Distance of the multi-variate distribution from the constraint manifold.

  The point on the manifold is given as, _{1 1} ð P þ P Þ þ P ð P þ P Þ

  ~x ¼ P ^{2 1 2 ^x 1 1 1 2 ^x 2} Simplifying we get, _{T 1} d P ½ ^{1 2 Š ð 1 þ P 2 Þ ½ 1 2 Š ð10Þ}

  ¼ ^x ^x ^x ^x

  it can be

  The details of simplifications are provided in Appendix observed that distance d is a weighted distance between the two data sources and it can provide a measure of nearness or farness of the two data sources to each other. A large value of d implies a large separation while a small d signifies closeness of the data sources. In other words, the distance from the manifold provides an indication of the relative disparity among the data sources.

  Theorem 3: For N dimension of n data sources, the d distance ( ₂ follow a chi-squared distribution with nN degrees of freedom (DOF), that is, d v ð Nn Þ.

  Proof: From

   ) we can write

_T

₁

  d ð Þ P ð Þ ð11Þ ¼ ^x ~x ^x ~x

  Applying Whitening Transformation, we get, _{T T 1 W W W W} ð Þ P ð ð Þ ð Þ

  ^x ~x ^x ~x Þ ¼ ^x ~x ^x ~x _{T T} ð12Þ ð ð Þ Þ ð ð Þ Þ ¼ y y where y ð Þ N 0; 1 ð Þ is an independent standard normal distribution. For N ¼ W ^x ~x _{P N 2} dimensions of state vector, the right-hand side of (

  is y , thus distance d _{i i 2} ¼1

  follows a chi-square distribution with N DOF, that is, d v ð Þ. For n data sources N with N states, ₂ d nN v ð Þ

  Since d is a chi-square distribution with nN DOF, then for any significance level ₂ a Nn 2 0; 1 ð Þ, v ð _a Þ is defined such that the probability, ₂ P d Nn v ð Þ ¼ a _a a

  Hence, to have a confidence of 100 1 ð Þ percent, d should be less than respective critical value. A value of a ¼ 0:05 is assumed in this paper unless specified. Chi-square table

  

  ] can be used to obtain the critical value for the confidence distance with a particular significance level and DOF.

  3.1 Inconsistency Detection and Exclusion To obtain reliable and consistent fusion results, it is important that the inconsistent estimates in a multisensor distributed system be identified and excluded before fusion.

  For this reason, at each time step when the fusion center receives computed estimates from sensor nodes, distance d is calculated. A computed distance d less than the critical value mean that we are confident about the closeness of sensor estimates and that they can be fused together to provide better estimate of the underlying states. On the other hand, a distance d greater than or equal to the critical value indicate spuriousness of the sensor estimates. At least one of the sensor estimate is significantly different than the other sensor estimates. To exclude the outliers, a distance from the manifold is com- puted for every estimate and compared with the respective critical values. For n sensor estimates the hypothesis and decision rule are summarized as follow,

  Hypotheses: H : ^x ^{1 2 n}

  ¼ ^x ¼ ¼ ^x H _{1 : ^x 1 2 n}

  6¼ ^x 6¼ 6¼ ^x ₂ Decision Rule: Accept H if d \ v ð Nn Þ _{2 a} Reject H if d v ð Nn Þ _a

  If the hypothesis H is accepted then the estimates are optimally fused using

   ) and

  (

  

On the other hand, rejection of null hypothesis means that at least one of the sensor

  estimate is significantly different than the other sensor estimates. The next step is to identify the inconsistent sensor estimates. A distance from the manifold is computed for each of the estimates as, _{T 1} d ð Þ P ð Þ; i _{i ¼ ^x i ~x ^x i ~x ¼ 1; 2; . . .; n i}

  The outliers are identified and eliminated based on the respective critical value, that is, ₂ if d _{i v ð Þ they are rejected. Where N is the dimension of individual data source. N a}

  3.2 Effect of Correlation on d Distance Since the estimates provided by sensor nodes in a distributed fusion architecture are correlated, it is important to consider the effect of cross-correlation in the calculation of confidence distance. The d distance for a pair of multivariate Gaussian estimates _{1 ; r 2 2 ; r , with cross-correlation r can be written as, 2 2}

  ^x and ^x _{1 2 12 2} ½ Š

  ^x ^{1 ^x 2} d ¼ ð13Þ _{2 2 2 2} r þ r r r _{1 2 12 21} It is apparent that the distance between the mean values is affected by the correlation between the data sources. Figure

  

  illustrates the dependency of confidence distance d

  shows the scenario in which a data source

  on the correlation coefficient. Figure (with changing mean and constant variance) is moving away from another data source (with constant mean and constant variance). The distance d is plotted for various values of correlation coefficients. The y-axis shows the percentage of rejection of the null hypothesis H . Figure

  

  shows the distance d with changing correlation coefficient from −1 to 1. It can be noted that ignoring the cross-correlation in distance d result in underestimated or overestimated confidence and may lead to incorrect rejection of true null hypothesis (Type I error) or incorrect retaining of false null hypothesis (Type II error). The proposed framework inherently takes care of any cross-correlation among multiple data sources in the computation of distance d.

  

Fig. 3. Effect of correlation on d distance (a) Percentage of rejecting the null hypothesis H with different correlation values (b) d distance with correlation q 2 ½ 1 ; 1 Š: Example: Consider a numerical simulation with the constant state, x ¼ 10 _k Three sensors are used to estimate the state x , where the measurements of the _k sensors are corrupted with respective variance of R ; R and R . The values for the _{1 2 3} parameters assumed in the simulation are,

  Q ¼ 2; R ^{1 ¼ 0:5; R 2 ¼ 1; R 1 ¼ 0:9}

  The sensors measurements are assumed to be cross-correlated. It is also assumed that the sensor 1, sensor 2 and sensor 3 measurements are independently affected by unmodeled random noise and produce inconsistent data for 33, 33 and 34% of the time respectively. The sensors compute local estimates of the state and send it to the fusion center. Three strategies for fusing the local sensor estimates are compared: (1) CP, which fuses the three sensor estimates using (

  with

  (2) CP WO-d means the outliers were identified and rejected based on ( ₂ r ¼ 0 before fusion, that is, correlation in computation of d is ignored and, (3) CP ₁₂ WO-dC, reject the outliers based on ( with taking into account the cross-correlation.

  Figure

   that

  neglecting the cross-correlation in CP WO-d result in Type II error, that is, all the three estimates are fused despite the fact that estimate 2 is inconsistent. CP WO-dC correctly

  

  identifies and eliminates the spurious estimate before the fusion process. Figure

  

Fig. 4. Three sensors fusion when the estimate of sensor 2 is inconsistent. Neglecting the

  

Fig. 5. Estimated state after three sensor fusion in presence of inconsistent estimates.

  shows the estimated state after fusion of three sensors estimates for 100 samples. It can be seen that the presence of outliers greatly affects the outcome of multisensor data fusion. As depicted in Fig.

  eliminating outliers before fusion can improve the esti-

  mation performance. The fused samples of CP WO-d and CP WO-dC on average lies closer to the actual state. Figure

   also shows the fusion performance when outliers are

  identified with and without cross-correlation. It can be noted that inconsideration of correlation affects the estimation quality because of Type I and Type II error.

4 Simulation Results

  In this section, simulation results are provided to demonstrate the effectiveness of the proposed method for fusion of spurious data. The performance is assessed by root mean square error (RMSE) over the simulation time computed as, _{X V s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2} 1 x i x i

  ð Actual ð Þ Estimated ð Þ Þ S ¼ _RMSE

  V L _{i ¼1} where L is the length of simulation and V is the Monte-Carlo runs. Consider a target tracking scenario characterized by the following dynamic system model,

  2

  1 T T =2 x ¼ x þ u þ w ð14Þ _{k k 1 k 1 k 1}

  1 T _T with the state vector x k ^{1 ¼ s v ½ Š , where s and v are the position and velocity of the} target at time t respectively. T is the sampling period and assumed as 3 s. The system process is affected by zero mean Gaussian noise w k ^{1 with covariance matrix Q. Three} sensors are employed to track the movement of the target, where the sensor mea- surements are approximated by the following equation,

  1 z k ¼ x k þ v k þ e k ; i ¼ 1; 2; 3 ð15Þ _{i i i}

  1 The measurements of the sensors are corrupted by noise v k with respective covariance _i of R i ; i ¼ 1; . . .; 3. The covariance of the process noise assumed is Q ¼ 10 and sensor measurement noises are,

  R R R _{1 ¼ diag 50; 30 ð Þ; 2 ¼ diag 70; 20 ð Þ; 3 ¼ diag 10; 60 ð Þ} The control input u _{k 1}

  ¼ 1 if v \ 30 otherwise it is changed to −1 until v \ 5: It is assumed that the sensor 1, sensor 2 and sensor 3 measurements are independently affected by unmodeled random noise e for 33, 33 and 34% of the time respectively _{k i} and thus the estimates provided by sensors are sometimes spurious.

  Starting from an initial value, in each time step the individual sensor uses local state prediction, that is, (

  to predict the state of the target and then update the state

  prediction by its own sensor measurements obtained through (

  The local estimates

  are assumed to be correlated and

   ) is used to calculate the track-to-track

  cross-correlation. The estimated states and covariances by each sensor are sent to the fusion center, where they are fused by CP Method, which takes care of the cross-correlation among the estimates. The three fusion strategies of CP (fusion without outlier removal), CP WO-d (outlier removal without considering cross-correlation) and CP WO-dC (taking care of correlation in outlier removal) are compared based on RMSE between the actual state value and fused estimate of the state for 1000 Monte Carlo runs. In the simulation setup, the inconsistency is detected with significance level a = 0.05. Figure

  and (b) illustrate the RMSE of the target position and velocity

  respectively versus time. Table

   summarizes the average RMSE for 1000 Monte Carlo runs.

  Figure

   and Table

  

  shows the efficacy of the proposed method in identifying and removing outliers. It can be observed that the presence of outliers deteriorates the fusion performance of multisensor data fusion. Eliminating the outliers before fusion greatly improve the estimation quality. Figure

  

also shows the fusion performance

  when outliers are identified with and without consideration of cross-correlation in dis- tance d. It can be noted that inconsideration of correlation affects the estimation quality because of Type I and Type II error.

  

Fig. 6. Illustration of distributed multisensor data fusion in presence of inconsistent estimates.

  

Table 1. Average RMSE for 1000 Monte Carlo runs

Average RMSE CP CP WO-d CP WO-dC

Position (m) 88.3793 50.7565 47.0373 Velocity (m/s) 29.5435 26.9081 25.0586