Ž .
ISPRS Journal of Photogrammetry Remote Sensing 55 2000 34–47 www.elsevier.nlrlocaterisprsjprs
Review Paper
Modelling and representation issues in automated feature extraction from aerial and satellite images
Arcot Sowmya
a,
, John Trinder
b
a
School of Computer Science and Engineering, The UniÕersity of New South Wales, Sydney NSW 2052, Australia
b
School of Geomatic Engineering, The UniÕersity of New South Wales, Sydney NSW 2052, Australia Received 6 October 1998; accepted 21 December 1999
Abstract
New digital systems for the processing of photogrammetric and remote sensing images have led to new approaches to Ž
. information extraction for mapping and Geographic Information System GIS applications, with the expectation that data
can become more readily available at a lower cost and with greater currency. Demands for mapping and GIS data are increasing as well for environmental assessment and monitoring. Hence, researchers from the fields of photogrammetry and
remote sensing, as well as computer vision and artificial intelligence, are bringing together their particular skills for automating these tasks of information extraction. The paper will review some of the approaches used in knowledge
representation and modelling for machine vision, and give examples of their applications in research for image understand- ing of aerial and satellite imagery. q 2000 Elsevier Science B.V. All rights reserved.
Keywords: artificial intelligence; computer vision; knowledge; models; photogrammetry; remote sensing; representation; feature extraction; automation
1. Introduction
Data acquisition by photogrammetry and remote sensing for mapping and GIS has traditionally been
characterised by the efficient manual extraction of high precision 3-D data from images ranging in scale
from those derived from space systems, to close- range images of stationary objects. These procedures
are generally time-consuming and expensive. This has limited the amount and resolution of terrain
information that can be extracted on current mapping and GIS budgets. In addition, in many cases, the data
Corresponding author. Tel.: q61-2-9385-3936; fax: q61-2- 9385-1814.
Ž .
E-mail address: sowmyacse.unsw.edu.au A. Sowmya .
has tended to become out-of-date because of poor revision cycles adopted by data acquisition authori-
ties. Data acquisition systems have always been tech- nology-driven, and have been adapted to the latest
methods and equipment. This will clearly continue in the future as digital systems become available. Digi-
tal systems provide opportunities for new approaches to information extraction for mapping and GIS based
on increasing levels of automation. They also accom- modate cutting-edge techniques in computer vision
that draw upon areas such as artificial intelligence and machine learning. Hence, researchers from the
fields of photogrammetry and remote sensing, as well as computer vision and artificial intelligence,
are blending their particular skills to attack the spe- cific problems in this application area.
0924-2716r00r - see front matter q 2000 Elsevier Science B.V. All rights reserved. Ž
. PII: S 0 9 2 4 - 2 7 1 6 9 9 0 0 0 4 0 - 4
Automatic information extraction of the terrain surface in the fields of photogrammetry and remote
sensing requires the formulation of procedures and knowledge that encapsulate the content of the im-
ages. This is a non-trivial task, because of the com- plexity of the information stored in the images.
Images of the terrain surface used in photogramme- try may have scales varying from 1:3000 to a larger
scale of 1:90,000, while in remote sensing the pixel footprints usually vary from 1 to 30 m. The structure
of the features in images of the terrain is complex, being a combination of many different intensities
that can represent natural features such as vegetation, geomorphological and hydrological features, objects
constructed by humans such as buildings and roads, and artifacts caused by variations in illumination of
the terrain by the sun, such as shadows and other changes in brightness. In addition, the context in
which features occur is considerably more complex than may occur in ‘normal’ photographic images.
These characteristics mean that extraction of infor- mation in aerial and satellite images presents major
challenges. The research on information extraction must consider primarily the semantic aspects of the
data. However, the geometric quality of the extracted information must also be considered, so that it satis-
fies the relevant specifications of spatial data.
Computer vision is the enterprise of automating and integrating a wide range of processes and repre-
Ž sentations used for visual perception Ballard and
. Brown, 1982 by constructing explicit, meaningful
descriptions of objects from images, using a variety of approaches and techniques including digital image
processing, pattern recognition, geometric modelling and cognitive processing. Computer vision concerns
itself chiefly with the problem of image interpreta- tion and understanding, and attempts to achieve it via
object and scene recognition. In this task, it employs the techniques of attribute and relation extraction
from an image, shape representation and description, and finally, model-based recognition using the at-
tributes extracted and described.
The application of computer vision approaches to the task of extraction of information from digital
remotely sensed images is complicated by the method of acquisition of these images, and the consequent
data characteristics need careful consideration when applying or adapting methods developed for ‘nor-
Ž .
mal’ photographic images De Gunst, 1996 . Many difficult-to-handle features present in computer vi-
sion applications, also tend to persist in aerial and satellite images. The common problems include noise
in the acquisition process, the effects of shading, variations in illumination and geometry due to cam-
era angle and position, and occlusion and partial objects problems.
There are differences in approaches taken in the interpretation of aerialrsatellite and other photo-
graphic images. Three-dimensional object geometry is an essential element of the processing of aerial
images, not only for the purposes of extracting eleva- tions, but also because the third dimension will
provide additional information for the interpretation of the image. The issue then arises as to whether 2-D
or 3-D information will result in a better feature description. 2-D image data has generally been con-
sidered sufficient for most research in the computer science community, though 3-D processing in the
form of stereoscopic vision and depth extraction are being addressed by some research groups. Object
models in computer vision are dominated by shape- and appearance-oriented descriptions, though there
are a few exceptions. For low- and medium-resolu- tion aerial images, the utility of shape-based recogni-
tion is questionable. However, context plays a larger role in remotely sensed image interpretation; for
example, a bridge may be more easily recognised as part of a road which crosses a river. It has been
observed that objects in an aerial image are dense and a composition of many parts; this agrees with
recent trends in computer vision, which recognise objects in images by first isolating component parts
of the objects and the relationships between them Ž
. Grimson, 1990 .
This paper will review some of the approaches used in knowledge representation and modelling for
machine vision and give examples of their applica- tions in research for image understanding of re-
motely sensed images. While a number of methods will be covered, it is not possible to discuss all
approaches currently being used by researchers in this field. Other surveys may be consulted for cover-
age of other areas, for example, Crevier and Lepage Ž
. 1997 for a survey of knowledge-based image un-
Ž .
derstanding, Hancock and Kittler 1990 for one on Ž
. relaxation techniques, and Srinivasan 1990 for a
survey of artificial intelligence techniques in remote sensing. Knowledge is defined in Section 2, as well
as approaches to knowledge representation, control issues and approaches to the modelling of features in
machine vision. Feature representation and the fea- ture recognition process are covered in Section 3,
while examples of the application of the methods of knowledge representation in both photogrammetry
and remote sensing are presented in Section 4.
2. Knowledge, representation and models
