Accuracy assessment of the laser data

Ž . Ž . Fig. 2. a Automatically detected breaklines from a laser-DTM image; b the corresponding manually measured breaklines. surface approximation methods. A simple model is to reconstruct a breakline as the intersection of two planes, computed by using the least squares method. Before modelling, some processing may be needed to close broken lines, and delete small lines as well as those lines that result from wrongly detected edges. This editing process is totally or partly man- ual. 4.2. Extraction of information from laser data in combination with existing databases In Section 4.1, it was clearly stated that laser data alone does not allow the extraction of all the infor- mation needed to generate a DSM for road planning and design. Furthermore, the quality of information may need to be improved. Thus, combination of laser with existing information is a prerequisite. In this section, only the combination of laser data with 2D and 3D Dutch databases is considered. No testing was carried out for the feasibility of such combina- tion. The databases considered are the 2D databases GBKN and Top10vector, and the 3D databases DTB-roads and DTM-roads. GBKN is a large-scale topographic base map, and Top10vector is a large- scale topologically structured vector database in scale 1:10000, generated by photogrammetric means. DTB-roads is a large scale database for road mainte- nance, produced by photogrammetric and terrestrial means, while DTM-roads is a database for road construction, generated by terrestrial means. Information stored in such databases can be used not only to label some of the features in the DSM, but also to locate them, e.g., boundaries of buildings or lines painted on roads. If the database is not object-oriented, as is the case of GBKN, before feature extraction, e.g., of buildings, from the points and lines, some processing has to be carried out to Ž group them into meaningful objects Lemmens et al., . 1997 . Their location in the laser data furnishes the height values for the objects in the database. This process, i.e., the projection of the objects or features in the database onto the laser data, requires some care. Their related semantic information should be used for a more adequate estimation of the heights. For objects on the terrain, like buildings, this estima- tion must be carried out using the raw measurements Ž . laser-DSM . Obviously, if the database is 3D and the heights are accurate, modelling may not be re- quired. Table 2 lists which of the information in Table 1 can be collected by combining laser data with the databases mentioned above. Since laser allows the extraction of the relief information, combination with databases is in principle needed solely to extract objects on the terrain.

5. Accuracy assessment of the laser data

As it can be observed from Table 2, the terrain relief can in principle be reconstructed by using the same features that are part of the photogrammetric DTM. Obviously, the procedures to extract all these features either automatically or manually are very demanding. Instead, the problem of reconstructing the terrain relief may be approached in another way. Considering that the required height accuracy of the Ž DTM for road planning and design is 25 cm 7.5 cm . for hard surfaces like roads , and the laser-DTM is quite dense, one should verify first if the laser-DTM allows the terrain to be reconstructed with an accu- racy comparable to that required. In this section, we estimate first both the plani- metric and height accuracy of the laser measure- ments. Then, the altimetric accuracy of the laser- DTM is computed. Accuracy is represented here by Ž . the root mean square difference RMSE between the laser and reference data. 5.1. Accuracy of the raw laser measurements The accuracy of the measurements is determined by the laser system, the process of measuring pro- cess, and the terrain itself. With respect to the laser system, the accuracy depends for instance on the signal-to-noise ratio of the received signal, the diver- gence of the laser beam, and the laser wavelength. Aspects of the measuring process that influence ac- curacy are for example, the viewing direction of the system, position and orientation errors of the plat- form, angle of incidence, transmission properties of the atmosphere, and the amount of background varia- tion. Regarding the terrain itself, the shape and spec- tral properties of land cover, and the slope with respect to the viewing direction also influence the Ž accuracy Heerd et al., 1997; Lemmens and Fortuin, . 1997 . Another factor that influences accuracy is the strategy adopted by the company to acquire and process the data. Of great importance is the way the adjustment of the several strips of laser data is carried out. Our experience of sub-contracting com- panies that acquire laser data shows that different companies have different adjustment methods, if any. Therefore, it is advisable that customers specify a proper adjustment procedure before data acquisition. 5.2. Altimetric accuracy The height accuracy of the laser measurements is strongly related to the accuracy of the laser system and to the terrain geometry. Because the latter is variable, and because its impact on the measured heights is also expressed in terms of the accuracy of the laser-DTM, the accuracy in altimetry is only estimated at locations where the terrain is generally Ž flat furthermore, the influence of errors in planime- . try will be minimised . This estimate is considered to be a fair approximation of the altimetric accuracy of the laser measurements. The points used for testing are not the original points as measured by the laser, but those bilinearly interpolated from the laser-DTM at the same locations as the reference measurements. Because only flat regions are used, the related inter- polation errors are very low. Three datasets are used to estimate the altimetric accuracy of the laser measurements: two sets are made of tachymetric measurements, and the third set is made of photogrammetrically measured points. The two sets of tachymetric measurements were carried out on bare soil and soil covered with low Ž . grass mean height 5 cm respectively, with an ex- Ž . pected accuracy RMSE of 3 cm. The photogram- metrically measured points were collected separately on roads covered with asphalt, and on bare soil and soil covered with low grass, with an expected RMSE of 7 cm. The obtained results are shown in Table 3. The RMSE values of 8 cm on roads and around 15 cm on fields with low grass and on bare soil Table 3 Altimetric accuracy of raw laser measurements Reference data Terrain type Number of Mean RMSE Ž . Ž . measurements error cm cm Tachymetric bare soil 362 y5 10 soil with low grass 397 y2 16 Photogrammetric paved road 344 y2 8 bare soil and soil with low grass 5336 y5 15 Ž . Ž . Fig. 3. a Manually extracted outlines of building roofs in a laser image; b the corresponding roof outlines collected photogrammetrically. corroborate the specified height accuracy of 15 cm Ž . Section 3 . 5.3. Planimetric accuracy The accuracy in X and Y of laser measurements should be computed by identifying the same points in the laser data and in the reference data. The most suitable points are the corners of building roofs. Obviously, the laser beam does not necessarily hit the corners of building roofs in the reference data. Thus, the planimetric accuracy can only be estimated roughly by modelling the roof boundaries. This is done by creating a grid for which the height values are estimated from the raw measurements in a neigh- bourhood. This grid can be converted into a laser image and the roof corners can then be measured manually. The grid spacing and neighbourhood sizes will influence the operator’s interpretation. This is also influenced by the method the height value as- signed to each pixel is computed within its neigh- bourhood. For this test, the height value of each pixel is estimated as the maximum height in the neighbourhood. As grid spacing and neighbourhood sizes, we chose the somewhat ad-hoc values of 50 cm = 50 cm, and of 80 cm = 80 cm. The final set of X and Y coordinates of 163 corners of the building roofs as measured by an operator in the laser images are then compared with those measured photogrammetrically with an accu- Ž . racy of 6 cm Fig. 3 . The variance of the differences thus computed is influenced by the measuring accu- racy of the operator. Therefore, the variance of the operator’s measurements is also estimated and sub- Ž tracted from that of the differences the measuring . accuracy of the operator is 14 cm . The final plani- metric accuracy of the laser measurements is found to be equal to 21 cm, whereas the mean error is 4 Table 4 Altimetric accuracy of the laser-DTM Terrain feature Number Mean RMSE Maximum Ž . Ž . Ž . of points error cm cm absolute error m Separation line between two paved surfaces 82 y4 15 1.04 Separation line between paved and unpaved surfaces 2099 y7 16 1.09 Lines painted on roads 344 y2 8 0.22 Protection walls: dam and quay 100 y15 24 1.01 Ditches 9757 8 36 4.18 Upper side of a sloped surface 1049 y25 38 1.97 Lower side of a sloped surface 729 15 28 1.00 String of points on flat surfaces 5336 y5 15 2.17 Individual points 127 6 18 0.66 Total RMSE 29 cm. As stated above, this estimate is a coarse esti- mate as it is influenced by extrinsic factors such as Ž . the grid spacing pixel size of the images and the operator’s interpretation. Nonetheless, it agrees with Ž . the specified planimetric accuracy Section 3 . 5.4. Altimetric accuracy of the laser-DTM Accuracy of the laser-DTM quantifies the fidelity of the laser data with respect to the terrain relief. It is influenced by the terrain characteristics, measuring accuracy, sampling spacing, and modelling function. Because it is difficult to identify the same points in the laser grid and the reference data, only the alti- metric accuracy is computed. This will as well re- flect the quality in planimetry of the laser-DTM. To get a good estimate of the altimetric accuracy of the laser-DTM for the purpose of road planning and design, the Z values along the terrain features that are part of the DTM in Table 1 are interpolated in the laser-DTM and compared with the correspond- ing values measured photogrammetrically. Ž . The results obtained Table 4 can be summarised as follows: for flat or near flat regions the altimetric accuracy of the laser-DTM varies from 8 cm to 15 cm. For sloped terrain, the RMSE varies from 25 cm to 38 cm. The RMSE of all the different features together is equal to 29 cm. This figure is slightly worse than that required for road planning and de- Ž . sign 25 cm . For sloped terrain, the height values are less accurate. If necessary, they may be improved by a more adequate modelling function than bilinear interpolation. As stated in Section 2, the accuracy of the photogrammetrically generated DSM has never been assessed. This means that the user may not know the minimum accuracy required or that his requirements in terms of accuracy are flexible. The need for improvement as well as better modelling should be a subject of further research.

6. Conclusions