1.3 Quality control and performance measures
How do we evaluate the quality of image orientation from a structure
from motion
algorithm? Of
course, each
implementation of SFM has its own quality optimization criteria as well as different parameters that control the various stages
from feature extraction, to feature matching, to image orientation. Changing the values of control parameters will
change the output of the process and to some extent its quality as well; how is the output quality to be evaluated?
In principle we can think of two sorts of rankings: from a user standpoint an absolute measure e.g. a sort of score of the
orientation goodness would perhaps be the best, since no other term of comparison another sequence oriented with a different
method is available; in a scientific and technical context, when comparing the results of different SFM strategies applied to the
same image sequence, relative scores
e.g. collinearity residuals, point redundancy, frame coverage, etc. might be acceptable. In
both cases, practically every quality measures depend to large extent on the data set network geometry, image quality, object
shape, calibration data, etc.. This means that both “absolute” and “relative” measures in fact apply to that specific dataset.
The goal of this paper is to provide some tentative proposal on this topic, that we believe should get more attention within the
photogrammetric community. In section 2 a brief description of the main blocks of our SFM strategy are presented; some details
about the current software implementation are also given. In section 3 some proposals are put forward on how to evaluate the
quality of automatic image orientation. In section 4, using datasets provided by the workshop organizers, the results of the
automatic orientation and the checks performed on three image sequences of cultural heritage sites where control data are
available are presented; besides, results and some statistics are presented on the orientation of other four image sequences also
retrieved from the organizer’s website.
2. OUR IMPLEMENTATION OF SFM