MULTI-SOURCE HIERARCHICAL CONDITIONAL RANDOM FIELD MODEL FOR FEATURE FUSION OF REMOTE SENSING IMAGES AND LIDAR DATA
Z. Zhang
a
, M.Y. Yang
b
, M. Zhou
a a
Key Laboratory of Quantitative Remote Sensing Information Technology, Academy of Opto-Electronics, Chinese Academy of Sciences, Beijing, China
zhangzheng, zhoumeiaoe.ac.cn
b
Institute for Information Processing TNT, Leibniz University Hannover, Germany yangtnt.uni-hannover.de
Commission WG VI4, WG III3
KEY WORDS: Feature fusion, Conditional Random Field, Image Classification, Multi-source Data, Hierarchical model ABSTRACT:
Feature fusion of remote sensing images and LiDAR points cloud data, which have strong complementarity, can effectively play the advantages of multi-class features to provide more reliable information support for the remote sensing applications, such as object
classification and recognition. In this paper, we introduce a novel multi-source hierarchical conditional random field MSHCRF model to fuse features extracted from remote sensing images and LiDAR data for image classification. Firstly, typical features are
selected to obtain the interest regions from multi-source data, then MSHCRF model is constructed to exploit up the features, category compatibility of images and the category consistency of multi-source data based on the regions, and the outputs of the
model represents the optimal results of the image classification. Competitive results demonstrate the precision and robustness of the proposed method.
1. INTRODUCTION
Nowadays, there are many different sources of earth observation data which reflect the different characteristics of targets on the
ground, so how to fuse the multi-source data reasonably and effectively for the application, such as object classification and
recognition, is a hot topic in the field of remote sensing applications. In all the data mentioned above, remote sensing
images and LiDAR points cloud have strong complementarity, so fusion of the two sources of data for object classification is
attached more and more attention and many methods were proposed. In general they can be classified into image fusion
Parmehr et al., 2012; Ge et al., 2012 and feature fusion Deng et al., 2012; Huang et al., 2011. The methods for image fusion
always include different resolution data sampling and registration, so the processing is time-consuming, and will
inevitably lose a lot of useful information, which reduces the accuracy of the subsequent image classification. In the feature
fusion methods, the features are usually extracted independently from different sources data, and the fusion lacks consideration
of correspondence of location and contextual information, so the classification results could be improved. In addition,
because the features selected in some methods are not invariant to rotation, scale, or affine, they are always poor in stability. In
order to overcome the shortages of former methods, this paper presents a novel multi-source hierarchical conditional random
field MSHCRF model to fuse features extracted from remote sensing images and LiDAR data for image classification. Firstly,
typical features are selected to obtain the interest regions from multi-source data. Then MSHCRF model is constructed to
exploit up the features, category compatibility of images and the category consistency of multi-source data based on the regions,
and the outputs of the model represents the optimal results of the image classification.
2. DESCRIPTION OF FEATURES SELECTED
In remote sensing images and LiDAR data, while the abundance of information offers more detailed information of interest
objects, it also enhances the noises. Selection of appropriate features in a reasonable way is important in our method.
In order to provide a reliable basis for subsequent processing, the proposed model contains five kinds of typical features: local
saliency feature LSF, line feature LF and texture feature TF are extracted from remote sensing images, mean shift feature
MSF and alpha shape feature ASF are from LiDAR data, so its robust to background interference, change of scale and
perspective, etc. The detector of KB Kadir et al., 2001 is a representative
LSF, which is invariant to viewpoint change, and sensitive to image perturbations. We utilize the detector of KB to
calculate saliency of each pixel in the images. LSD is a linear-time line segment detector that gives accurate
results, a controlled number of false detections, and requires no parameter tuning. In accordance with the method introduced in
Grompone et al., 2010, we can calculate the response value at each pixel.
As the basic unit of TF, Texton is utilized to distinguish between foreground and background regions effectively and
increase the accuracy of the results. Similar to the method in Shotton et al., 2009, we can obtain the response to Texton of
each pixel in the image. For the sparseness and discreteness of LiDAR points cloud data,
we utilize an adaptive mean shift algorithm which is a sample International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences,
Volume XL-1W1, ISPRS Hannover Workshop 2013, 21 – 24 May 2013, Hannover, Germany
389
point estimation method based on data-driven. In our model, the specific process of achieving the MSF is introduced in
Georgescu et al., 2003. Based on the planar features obtained, the alpha shape
algorithm is used to extract the boundary contour of each target, and then the Delaunay triangulation is used to get the line
feature of LiDAR points cloud. The extraction of the ASF refers to Shen et al., 2011.
3. FEATURE FUSION USING MSHCRF
