information along with depth information for each point. It is possible to use color information to improve the robustness of the
segmentation performance. In this paper, a color-enhanced hybrid segmentation model based
on the region growing method is proposed for RGB-D camera- based indoor mobile mapping point cloud planar surface
segmentation, and the model is more robust to the clustered, noisy and incomplete point cloud data compared to the traditional
point cloud segmentation method. The model combines the color-based information with the curvature-based information for
the seed point selection in the region growing method. A hybrid growing criteria is also developed with consideration of the color
similarity, curvature similarity and point continuity. The hybrid weight is determined by a segmentation evaluation processing
based on a small set of labeled segmented data. The segmentation results are given based on the hybrid weight effect on the
segmentation performance comparison between the segmentation methods.
2. DATA ACQUISITION AND PRE-PROCESSING
The indoor mobile mapping system is with a weight of 22.5 kg and with a size of 46 cm × 50 cm × 82 cm length × width ×
height. The basic mobile platform is four-wheeled Pioneer3-AT robot. The system is equipped with a RGB-D camera Kinect
sensor, 640 × 480 pixels, and 57
o
× 43
o
field-of-view for 3D range data, and the camera acquires 3D range data under various
illumination situations because it illuminates the object based on infrared radiation. A 2D laser scanner SICK LMS100, which
covers a scanning area of 270
o
, is mounted on the platform to achieve 2D scan profile for 2D map building. The mapping
system is capable of operating 4 hours with three full charged batteries 12 V lead acid, 7.2 AH, and its core system is an Intel-
i5-2.53 GHz processor and 2 GB RAM with a Linux operating system As shown in figure 1.
Figure 1. Mobile mapping system design Since the point cloud data acquired by the RGB-D camera-based
system are limited Han et al., 2013, a pre-processing process is needed shown in figure 2. The point cloud pre-processing
method used in this paper include: 1 down-sampling for acquiring point clouds with consistent resolution, 2 de-noising
using Gaussian filtering Liu et al., 2012, 3 point cloud data interpolation using the moving least squares MLS smoothing.
We determined that with the down-sampling, the point cloud density decreased dramatically but are accompanied by low
resolution and an uneven distribution. Finally, the points are evenly distributed after MLS smoothing.
Down-sampling Denoising
Original point cloud
c b
4632879 670094
663390 678545
MLS smoothing a
Figure 2. Point cloud data pre-processing. a Original Point cloud. b Points number. c Close look of the point cloud
3. COLOR-ENHANCED HYBRID SEGMENTATION
MODEL
For the original region growing segmentation algorithm, only the curvature information is used for seed selection. With the quality
problem of the RGB-D camera-based indoor mobile mapping point clouds data, a more robust seed selection method and
growing criteria are required. We combine the color moment features with the curvature feature for the seed point selection
and growing criteria, and use a segmentation evaluation process to optimize the hybrid weight.
3.1 Color-enhanced Seed Point Selection
For each point in the point cloud, select k neighbor points of radius and calculate the covariance matrix as:
Cov Cov ∙
∙ , ∈ , , 1 where = centroid position of the
k neighbor points of = the
feature value in the covariance matrix = the
feature vector in the covariance matrix The smallest component of refers to the normal vector of .
The curvature of is expressed as:
2 The first-, second- and third-order moments of the color feature
of in the radius of r is calculated as:
∑ ,
, , 3 ∑
, , , 4
∑ ,
, , 5 where
= the first-order moment of the hue, saturation and illumination components of the HSV color
space of = the second-order moment of the hue,
saturation and illumination components of the HSV color space of
= the third-order moment of the hue, saturation and illumination components of
the HSV color space of = one of the color channels of the
nth neighbor point of
ISPRS Technical Commission I Symposium, 17 – 20 November 2014, Denver, Colorado, USA
This contribution has been peer-reviewed. The double-blind peer-review was conducted on the basis of the full paper. doi:10.5194isprsannals-II-1-61-2014
62
When the features of the normal vector, curvature and color moments for each point are computed, the seed point for region
growing is selected based on two principles: 1 the color-stable region and 2 the geometry-stable region. As shown in figure 3,
the steadiness of color feature is represented by the second-order moment
∑ . The steadiness of the geometric shape is represented by the curvature
. The unstable value of the candidate seed point considering the color and curvature
information, , is represented as:
∙
∑
∙ 6
where = hybrid weight of the segmentation model = curvature of the point
∑ = second-order moment of the color
feature in HSV color space A smaller unstable value of the point indicates improved stability
of the point in the range. Here, the point with the minimum unstable value in the neighborhood is selected as the initial seed
point of region growing.
a b
c d Figure 3. Example of the point cloud curvature and the second-
order moments of the color map color. a Original point cloud. b Corresponding curvature distribution. cThe second-order
moment of the color feature Hue element. d Hybrid result. Example of the point cloud curvature and the second-order
moments of the color map color is given in figure 3. The red portion represents the more stable points in the region, while the
blue portion represents the more unstable points in the region. The seed point selected in the original curvature-based region
growing method figure 3b is improved by the color moment- based feature figure 3c, and the hybrid of the color and
curvature features helps in the selection of a more stable seed point figure 3d.
3.2 Growing Criteria
