3. BACKGROUND
A metrological characterization of this type of sensors have been proposed by different authors in the last few years, making
evident the high variability of the 3D results, and sometimes also of the excessive variability of the related characterization. Some
researcher attempted for example to define the optimal operating distance for Primesense-based Kinect obtaining 1000-3000 mm
as the range when with best overall performances operating with the OpenKinect framework Khoshelham and Elberink, 2012,
800-1500 mm operating with the SDK Alnowami et al., 2012 and 700-2000 mm operating with the OpenNI framework
Molnár et al., 2012. A comparison between two Primesense-based devices as the first
Kinect and the Asus Xtion presented similar result Gonzalez- Jorge et al., 2013. According to this paper both exhibited
negligible variations acquiring the same test object from different angles, showing instead very large accuracy changes from 5mm
to 15mm, and from 5mm to 25mm at a sensor-to-target distance of 1m and 2m respectively. In addition, this analysis gave a
precision data ranging from 1mm to 6mm for 1m range and from 4mm to 14mm for 2m range. Probably the high variability in the
latter results is also due to the small size of the calibrated object with the respect of the limited spatial resolution of the analyzed
devices, and in general to the relatively limited number of samples involved in this type of analysis, that make them more
prone to high variability of the related characterization results. 3.1 Spatial and temporal error
In general noise in a depth image is either Spatial – i.e. observable within a single frame - or Temporal – i.e. observable across
multiple consecutive frames - Mallick et al., 2014. Even if both make sense when characterizing a 3D device, in the practical
application a spatial characterization allows to obtain more statistically significant results even with just one isolated 3D
image. This is the approach we have followed for obtaining some metrological parameters in function of the operating range, that
could be practically usable for estimating accuracy and precision of each analysed device.
Provided that one 3D frame is acquired in a fraction of a second, focusing our attention on the spatial error allowed also to neglect
the effects of temperature in time, that have been proved to influence the performances of this range of devices over a time-
span of several tens of minutes DiFilippo and Jouaneh, 2015. The aim of the performed tests is therefore more on the relative
differences between the various points belonging to the same range image rather then the absolute deviation between such
coordinates and their true values. 3.2 Systematic and random error
According to the International Vocabulary of Metrology Joint Committee For Guides In Metrology JCGM, 2008, the
measurement uncertainty of each measurement equipment or device represents the cumulative effect of the systematic errors –
associated to the concept of accuracy –, that once modelled can be eliminated through a proper calibration, and unavoidable
random errors – associated with the concept of precision –, dependant by unpredictable causes like the electronic noise, that
can be only statistically characterized for making the end-user aware of the measurement system intrinsic limitations. Such
quantity can be evaluated by 3D digitizing a certified object, estimating the deviation of each measured value from the
theoretical one and calculating the standard deviation of. Such value represents an estimation of the global measurement
uncertainty in that operating conditions.
4. MATERIAL AND METHODS 4.1 Devices
Five actual devices have been tested featuring three different technologies: i Kinect V1 by Microsoft, Structure Sensor by
Occipital, and Xtion PRO by ASUS, all based on different implementations of the Primesense sensor; ii F200 by
IntelCreative, implementing the Realsense pattern projection technology; Kinect V2 by Microsoft, equipped with the Canesta
TOF Camera. 4.2 Control software
The software used for controlling the devices has been the following: i Skanect
http:skanect.occipital.com for all
Primesense-based devices i.e. Kinect1, ASUS Xtion and Structure Sensor; ii KScan3D
http:www.kscan3d.com for
the Kinect2; iii Camera Explorer https:software.intel.comen-
usblogs20150319realsense-sdk-and-camera-setup-on- windows-10
for the Realsense F200. While Skanect has been used on both Windows and Mac
platforms using indifferently USB 2 or 3, the other two packages were just for use on a Windows 8 or higher systems, connected
to the sensor through a USB3.
Figure 2. Devices characterized during the tests described in this paper.
4.3 3D processing software
All the raw 3D data has been exported as mesh generated on the acquired points and saved in Wavefront .obj format. When not
possible i.e. for the Kinect2 system, the data have been exported as ASCII file of 3D coordinates.
When the mesh was not natively available from the sensor, the unstructured cloud has been triangulated with the “IMAlign”
module of Innovmetric Polyworks, avoiding any smoothing during the mesh generation.
All the other 3D processing actions have been conducted with the open-surce packages Meshlab and Cloud Compare, and with the
commercial package Innovmetric IMInspect. Finally, additional statistical analysis has been made on
MATLAB with custom scripts developed within this project. 4.4 Test object
The main test object is the reference plane, that was made with a rectangular piece of float glass with a thickness of 11mm. The
dimensions of the rectangle are 700 mm × 528 mm, appropriate for covering a significant part of the framed area by each of the
sensors under test in the different working conditions. The reason behind using the glass is because it is very low cost and the
surface is very smooth with small deviations from the theoretical plane. The float glass manufacturing process allows to obtain a
planar target with a peak deviation from the theoretical plane in the order of few micrometers. Geometrically, this is suitable for
testing the devices under observation, characterized by The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLI-B5, 2016
This contribution has been peer-reviewed. doi:10.5194isprsarchives-XLI-B5-33-2016
35
measurement uncertainties in the range of millimeters. This material is obviously not usable “as is” due to its transparency,
not compliant with an active range sensing device made for working with a diffusive reflecting surface. This problem was
resolved by painting the surface matt white, using the painting process employed in the car industry, that ensure a uniform
distribution of paint not distorting the geometry of the plane. With this solution the final surface of the painted side has almost
the same surface qualities that of the glass with nicely diffusive reflecting surface Guidi et al., 2010.
Tests with reference planes at different operating distances have been made for extracting two crucial parameters: the systematic
error and the random error Guidi, 2013. 4.5 Systematic and random error estimation method
The method here employed is based on simple considerations considering the 3D digitizer as a “black box” system whose
behavior influence the 3D cloud differently at different spatial frequencies. In particular, we know that the random error is
largely due to the thermal noise of the electronic stage capturing the IR images, and by the laser speckle originated by the material
response to coherent light. All these effects are superimposed on the deterministic content of the probing pattern, whose image is
used for estimating the parallaxes in the different points of the frame. In this paper we assume to model the global random
contribution of each range image as Additive White Gaussian Noise AWGN, similarly to the thermal noise affecting CCD and
CMOS sensors. The fact that this contribution can be considered additive with good approximation is confirmed by the literature
Hirakawa and Parks, 2006. This means assuming to consider three separate contributions for each depth sample collected by
the sensor: i the actual measured value; ii the random error; iii the systematic error. The sum of the latter two define the spatial
measurement uncertainty. Actually all these contributions are delivered simultaneously by
the sensor, and the purpose of the technique here proposed is to discriminate them with low-cost equipment easily replicable in
any small laboratory. In Figure 3, representing the color-coded deviation of each z value from the fitting plane associated with a
set of points all nominally belonging to the same plane, this superposition is made evident.
Figure 3. Color-coded deviation from the best-fitting plane originated by a range map acquired with the ASUS
Xtion device while capturing the reference plane located at 650mm by the sensor. The color scale
ranges from -12mm pink to +12mm yellow.
The “grainy” appearance of the mesh represented in Figure 3 indicates the presence of a random error component, that
typically is associated to high spatial frequencies, while the evident green spot in the center positive errors, with the bluish
area in the peripheral area of the image negative errors, show a systematic distortion of the measured values.
Thanks to the strong difference between the two components in terms of spatial frequency, we applied a low-pass filtering to the
cloud, with a set of parameters such to leave only the low- frequencies associated to the systematic errors.
The spatial filtering was achieved with the λ-µ smoothing algorithm Taubin, 1995, implemented in Meshlab, using λ=1,
µ=-0.53 and 50 iterations. This parametrization allowed to filter out all the high frequency components that we associate to the
random errors. The output of this step, shown in figure 4, represents a good approximation of the systematic error affecting
the range map shown in Figure 3.
Figure 4. Color-coded deviation from the best-fitting plane of the same same map shown in Fig. 3, after the application
of a low-pass filter leaving the systematic 3D distortion only. The color map is the same of Fig. 3.
The last step for estimating the random component of the range measurement error is to compare the raw 3D point cloud shown
in Figure 3, with its smoothed version shown in Figure 4. Such deviation, as visually confirmed by its color coded representation
in Figure 5, is far more uniformly distributed along the whole image area.
Figure 5. Color-coded deviation of the raw range map from the filtered one. The color map is the same of Fig. 3. The
error span is clearly reduced and the error appears more randomly distributed over the plane.
In conclusion, for each 3D acquisition of the reference plane, we can estimate the standard deviation of each acquired point from
the best-fitting plane. This represents an estimation of the global spatial uncertainty σ
u
. The 3D same cloud is then low-pass filtered and meshed. The standard deviation between the raw 3D
data and the filtered mesh represents an estimation of the spatial random error σ
r
. Since the uncertainty is the global parameter, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLI-B5, 2016
This contribution has been peer-reviewed. doi:10.5194isprsarchives-XLI-B5-33-2016
36
once the random component is estimated, their difference will represent an estimation of the systematic component of the error.
5. EXPERIMENTAL RESULTS
Kategori