Intensity filter and point clustering

2.2 Intensity filter and point clustering

The surface of any traffic sign incorporates sheeting with retroreflectivity materials. Therefore, light is redirected from the sign surface to the source and traffic signs are still visible at night McGee, 2010. The intensity attribute of a 3D point cloud acquired with a laser scanner is proportional to the reflectivity of the objects, and it is a property that can be used to distinguish highly retroreflective objects from their surroundings González- Jorge et al., 2013. Once the ground is removed from a point cloud, a Gaussian Mixture Model GMM with two components is estimated using the intensity distribution of all the points in the cloud. The points from the component with biggest mean are considered as retroreflective points, whereas the remaining points are removed from the cloud. This step performs a finer intensity filter than the previous step where the intensity based raster image � � was used. The remaining point cloud after the application of this filter may comprise objects made of retroreflective materials, such that traffic signs, license plates, metallic parts in buildings that are relatively close to the laser scanner or even pedestrian reflective clothing. The points of these objects, however, still remain unorganized. It is necessary to group together points that belong to the same object in order to analyse them separately. For that purpose, DBSCAN algorithm Ester et al., 1996 is used. It clusters close points with a certain density and marks points in low density areas as outliers. This way, the points in the cloud are organized and any noise remaining after the previous filtering stages is removed. Finally, it is possible to filter out those clusters that do not follow the geometric specifications for vertical traffic signs, which are previously known. Only planar clusters whose height is between 0.3m and 5m are kept. The planarity of a cluster is defined as: � � = √� − √� √� 2 where � , � , � are the eigenvalues of the covariance matrix of the points within the cluster. Following the criteria of Gressin et al. 2013, a cluster of points is considered as planar if � � . This removes non-planar clusters, while the height restriction filters objects such as license plates that could be a source of false positives Fig 2b. 2.3 Geometric inventory At this point of the process, it is assumed that each cluster represents a traffic sign. Therefore, the organized point cloud data can be analysed and several features of interest can be retrieved. Note that the traffic sign detection process relies on the intensity attribute of the 3D points and therefore in the retroreflectivity properties of the traffic sign surface. The detected clusters do not contain traffic sign poles even though they should be taken into consideration for the inventory together with the sign panel, as they can be bent or inclined. Figure 2. a Ground segmentation. The points in the ground painted in brown are filtered from the point cloud. b Intensity filter. Points of reflective objects painted in red are selected. Before obtaining any parameter, each cluster is analysed in order to determine if the traffic sign is held on a pole. For that purpose, a region growing approach reconstructs the sign panel surroundings, and an iterative pole-like object detection is performed, searching for a group of points under the traffic sign panel whose linearity � � . , being � � = √� − √� √� 3 If a pole is found, its director vector is defined as the eigenvector corresponding with the biggest eigenvalue � for the points which scored the largest � � value in the iterative process, and the inclination angle of the pole is computed in both front view � and profile view � � of the sign. Furthermore, and independently of the existence of a pole holding the sign, the following parameters are extracted from each cluster:  Position � , a x,y,z point defined as the centroid of the points of the traffic sign panel.  Height ℎ of the traffic sign with respect to the ground.  Azimut � . Clockwise angle between the projection of the normal vector of the sign panel on the XY plane and the North.  Contextual parameters, namely the distance and the angle � between the traffic sign and the trajectory. Finally, a vector with the parameters �� � = � , ℎ , � , , � , � , � � is assigned to each cluster of points, and defines the geometry of the traffic sign. Fig. 3 These data may be used to update any existing database and to be compared with the information collected in previous surveys in order to detect changes in the geometry of the sign. 2.4 Point cloud projection on images and traffic sign recognition The geometric parameters defined in the previous step do not offer the complete meaning of a traffic sign. The specific class of each sign must be retrieved from the MMS data together with the geometric inventory. For that purpose, the 3D points of each traffic sign are projected into 2D images taken during the survey. This contribution has been peer-reviewed. doi:10.5194isprsarchives-XLI-B3-717-2016 719 Every 2D image has associated with it the position and orientation of the vehicle, and the GPS time in the moment each Figure 3. Geometric inventory. Different parameters can be extracted using the point cloud data a Front view of the traffic sign. b Lateral view of the traffic sign. b Contextual features using the trajectory. image is taken, which is synced with the 3D point cloud. The usage of 2D images is motivated by the aforementioned insufficient resolution of laser scanners for the traffic sign recognition task. The inputs for this process are the traffic sign clusters which were obtained in Section 2.2 and contain the coordinates and time stamp for each point of the sign, the 2D images, the data associated with each image mainly time stamp and vehicle frame for every image, and the orientation parameters of the four MMS cameras with respect to the vehicle frame. Let = , , , be a traffic sign cluster. First, only those vehicle frames whose time stamp is in the range of ̅ ± are selected. Subsequently, the points in the cloud are transformed from the global coordinate system to each vehicle coordinate frame. This way, both the 3D cloud and the coordinate system of each camera = … are measured from the same origin. The camera plane � and the principal point of the camera are defined as: � ≡ � · [ , , − , , ] = 4 = − , , � � 5 where � � is the principal axis, is the focal distance and , , is the optical centre. Finally, points in are projected to � . The projection on camera coordinates , of each point is transformed to pixel coordinates � , � : � = ′ + � 6 � = ′ + � 7 where ′ , ′ is the distorted image point, considering radial distortion, , is the principal point of the camera and � is the pixel size. This process is graphically described in Fig. 4a. With this projection, the region of interest ROI of one or several images which contains the same traffic sign can be precisely defined. In order to add some background to the ROI and minimize possible calibration errors, a margin of 25-30 of the ROI size is added to the initial region Fig 4b. Once the images that contain a traffic sign are stored, a Traffic Sign Recognition TSR algorithm can be applied. There already exist numerous TSR methods which have proven remarkable results as mentioned in Section 1. Since this work is focused on 3D point cloud processing and not on image processing, the TSR method used will be briefly outlined. For each image, two colour – red and blue – bitmaps are created, based on the pixel intensity of the image in Hue-Luminance- Saturation HLS space. The shape of the binary image is classified in seven classes that represent traffic sign categories prohibition, indication… using Histogram of Oriented Gradients HOG Dalal and Triggs, 2004 as feature vector and Support Vector Machines SVM Cortes and Vapnik, 1995 as classifier. Given the sign category, a second classification is performed. A colour-HOG feature is used in this case, which is created by concatenating the HOG feature of each colour channel of the image in the CIELab colour space Creusen et al., 2010. This feature is again classified by means of SVM, and the type of the sign is finally retrieved. Figure 4. Point cloud projection on images. a The points of a traffic sign panel are projected into the camera plane. The distances of the projected point with respect to the principal point of the camera are transformed to pixel coordinates. b The bounding box of the projected points red rectangle is extended green rectangle in order to add some background to the detection, and the image is cropped. This contribution has been peer-reviewed. doi:10.5194isprsarchives-XLI-B3-717-2016 720

3. RESULTS