Academic Press Computer Vision And Applications May 2000 ISBN 0123797772 pdf
Computer Vision
and Applications
A Guide for Students and Practitioners LIMITED WARRANTY AND DISCLAIMER OF LIABILITY
ACADEMIC PRESS ("AP") AND ANYONE ELSE WHO HAS BEEN INVOLVED IN THE CREATION
OR PRODUCTION OF THE ACCOMPANYING CODE ("THE PRODUCT") CANNOT AND DO
NOT WARRANT THE PERFORMANCE OR RESULTS THAT MAY BE OBTAINED BY USING THE
PRODUCT. THE PRODUCT IS SOLD "AS IS" WITHOUT WARRANTY OF MERCHANTABILITY
OR FITNESS FOR ANY PARTICULAR PURPOSE. AP WARRANTS ONLY THAT THE MAGNETIC
CD-ROM(S) ON WHICH THE CODE IS RECORDED IS FREE FROM DEFECTS IN MATERIAL
AND FAULTY WORKMANSHIP UNDER THE NORMAL USE AND SERVICE FOR A PERIOD OF
NINETY (90) DAYS FROM THE DATE THE PRODUCT IS DELIVERED. THE PURCHASER’S SOLE
AND EXCLUSIVE REMEDY IN THE VENT OF A DEFECT IS EXPRESSLY LIMITED TO EITHER
REPLACEMENT OF THE CD-ROM(S) OR REFUND OF THE PURCHASE PRICE,AT AP’S SOLE
DISCRETION.
IN NO EVENT,WHETHER AS A RESULT OF BREACH OF CONTRACT,WARRANTY,OR TORT
(INCLUDING NEGLIGENCE),WILL AP OR ANYONE WHO HAS BEEN INVOLVED IN THE CRE-
ATION OR PRODUCTION OF THE PRODUCT BE LIABLE TO PURCHASER FOR ANY DAM-
AGES,INCLUDING ANY LOST PROFITS,LOST SAVINGS OR OTHER INCIDENTAL OR CON-
SEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PRODUCT
OR ANY MODIFICATIONS THEREOF,OR DUE TO THE CONTENTS OF THE CODE,EVEN IF
AP HAS BEEN ADVISED ON THE POSSIBILITY OF SUCH DAMAGES,OR FOR ANY CLAIM BY ANY OTHER PARTY.
ANY REQUEST FOR REPLACEMENT OF A DEFECTIVE CD-ROM MUST BE POSTAGE PREPAID
AND MUST BE ACCOMPANIED BY THE ORIGINAL DEFECTIVE CD-ROM,YOUR MAILING AD-
DRESS AND TELEPHONE NUMBER,AND PROOF OF DATE OF PURCHASE AND PURCHASE
PRICE. SEND SUCH REQUESTS,STATING THE NATURE OF THE PROBLEM,TO ACADEMIC
PRESS CUSTOMER SERVICE,6277 SEA HARBOR DRIVE,ORLANDO,FL 32887,1-800-321-
5068. AP SHALL HAVE NO OBLIGATION TO REFUND THE PURCHASE PRICE OR TO RE-
PLACE A CD-ROM BASED ON CLAIMS OF DEFECTS IN THE NATURE OR OPERATION OF
THE PRODUCT.
SOME STATES DO NOT ALLOW LIMITATION ON HOW LONG AN IMPLIED WARRANTY
LASTS,NOR EXCLUSIONS OR LIMITATIONS OF INCIDENTAL OR CONSEQUENTIAL DAM-
AGE,SO THE ABOVE LIMITATIONS AND EXCLUSIONS MAY NOT APPLY TO YOU. THIS WAR-
RANTY GIVES YOU SPECIFIC LEGAL RIGHTS,AND YOU MAY ALSO HAVE OTHER RIGHTS
WHICH VARY FROM JURISDICTION TO JURISDICTION.
THE RE-EXPORT OF UNITED STATES ORIGINAL SOFTWARE IS SUBJECT TO THE UNITED
STATES LAWS UNDER THE EXPORT ADMINISTRATION ACT OF 1969 AS AMENDED. ANY
FURTHER SALE OF THE PRODUCT SHALL BE IN COMPLIANCE WITH THE UNITED STATES DEPARTMENT OF COMMERCE ADMINISTRATION REGULATIONS. COMPLIANCE WITH SUCH REGULATIONS IS YOUR RESPONSIBILITY AND NOT THE RESPONSIBILITY OF AP.
Computer Vision
and Applications
A Guide for Students and Practitioners
Editors
Bernd Jähne
Interdisciplinary Center for Scientific Computing University of Heidelberg,Heidelberg,Germany and
Scripps Institution of Oceanography University of California,San Diego
Horst Haußecker
Xerox Palo Alto Research Center San Diego San Francisco New York Boston
London Sydney Tokyo This book is printed on acid-free paper. Copyright © 2000 by Academic Press.
All rights reserved.
No part of this publication may be reproduced or transmitted in any form or
by any means, electronic or mechanical, including photocopy, recording, or
any information storage and retrieval system, without permission in writing
from the publisher.
Requests for permission to make copies of any part of the work should be
mailed to: Permissions Department, Harcourt, Inc., 6277 Sea Harbor Drive,
Orlando, Florida, 32887-6777.ACADEMIC PRESS
A Harcourt Science and Technology Company 525 B Street, Suite 1900, San Diego, CA 92101-4495, USA http://www.academicpress.com Academic Press 24-28 Oval Road, London NW1 7DX, UK http://www.hbuk.co.uk/ap/ Library of Congress Catalog Number: 99-68829 International Standard Book Number: 0–12–379777-2 Printed in the United States of America
Contents xi
27 . . . . . . . . . . . . . . . . .
80 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
75 . . . . . . . . . . . . . . . . . . . . . . . .
67 . . . . . . . . . . . . . . . . . . . . . . . .
66 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
56 . . . . . . . . . . . . . . . . . . . . .
54 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54 . . . . . . . . . . . . . . . . .
53 P. Geißler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
46 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31 . . . . . . . . . . . . . . . . . . . . . . . . .
17 . . . . . . . . . . . . . . .
xv
13 . . . . . . . . . . . . . . . . . . . . . . . . .
12 . . . . . . . . . . . .
11 H. Haußecker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 . . . . . . . . . . . . . . .
3 . . . . . . . . . . . . . .
2 . . . . . . . . . . . . . . .
1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1 B. Jähne . . . . . . . . . . . . . . . . . . .
84 vi
Contents
5 Solid-State Image Sensing 111
6 Geometric Calibration of Digital Imaging Systems 153
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
. . . . . . . . 155
. . . . . . . . . . . . 157
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
. . . . . . . . . . . . . . . 163
. . . . . . . . . . . . . . . . . . . . 170
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
R. Schwarte, G. Häusler, R. W. Malz
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
. . . . . . . . . . . . . . . . . . . . 179
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
. . . . . . . . . . . . . . 196
. . . . . . . . . . . . . . . . . . . . . . 199
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
. . . . . . . . . . . . . . . . . . . . . . . . 154
R. Godding
85 H. Haußecker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
P. Seitz
85 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
86 . . . . . . . . . . . . . . . . . . . . . . . . . .
88 . . . . . . . . . . . . . . . . . . . . . . . . . . .
91 . . . . . . . . . . . . . . . . . . . . . . . . . . . .
94
. . . . . . . . . . . . . . . . . . . . . . . . . . 108
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
. . . . . . . . . . . . 113
. . . . . . . . . . . . . . . . . . . . . . . . 120
. . . . . . . . . . . . . . . . . . . . 127
. . . . . . . . . . . . . . . . . . . . . . . 130
. . . . . . . . . . . . . . . . . . . . 134
. . . . . . . . . . . . . . . . . . . . 139
. . . . . . 146
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
7 Three-Dimensional Imaging Techniques 177
Contents
. . . . . . . . . . . . . . . . . . . . . . . . . 382
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
347
H. Haußecker and H. Spies
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
. . . . . . . . . . . . . . . . . . . 349
. . . . . . . . . . . . . . . 358
. . . . . . . . . . . . . . . . . . . . . 372
. . . . . . . . . . . . . . . . . . . . . . . 379
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
397
P. Geißler, T. Dierig, H. A. Mallot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
439
J. Weickert. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
. . . . . . . . . . . . . . . . . . . . . . . . . . . 448
. . . . . . . . . 335
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
vii
. . . . . . . . . . . . . . . . 246
211
B. Jähne
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215
. . . . . . . 224
. . . . . . . . . . . . . . . 232
. . . . . . . . . . . . . . . . . 237
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
. . . . . . . . . . . . . . . . . . . . . . . . 260
. . . . . . . . . . . . . . . . . . . . . . . 267
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
273
B. Jähne
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
. . . . . . . . . . . . . . . . . . . . . . 278
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
. . . . . . . . . . . . . . . . . . . . . . . 292
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
viii
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
541
H. Haußecker and H. R. Tizhoosh
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
. . . . . . . . . . . . . . . . . . . . . . 548
. . . . . . . . . . . . . . . . . . . 553
. . . . . . 556
. . . . . . . . . . . . . . . . . . . . 564
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 570
577
. . . . . . . . . . . . . . . 538
A. Meyer-Bäse
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577
. . . . . . . . . . . . . . . . . . . . . . 579
. . . . . . . . . . . . . . . . . . . 585
. . . . . . . . . . . . . . . . 590
. . . . . . . . . 593
. . . . . . . . . . . . . . . . . . . . . . . 596
. . . . . . . . . . . . . 601
. . . . . . . . . . . . . . . . . . . . . . . . . . . 604
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
Contents
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
459
C. Schnörr
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459
. . . . . . . . 463
. . . . . . . . . . . . . . . . . 474
. . . . . . . . . . . . . . . . . . . . 476
483
P. Soille. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484
. . . . . . . . . . . . . . . . . . . . 489
. . . . . . . . . . . . . . . . . 495
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
517
J. Hornegger, D. Paulus, and H. Niemann. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
. . . . . . . . . . . . . . . . . . . . . . . 518
. . . . . . . . . . . 519
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605
Contents
Th. Hermes and O. Herzog . . . . . . . 632
H.-G. Maas . . . . . . . . 652
H. Spies, H. Gröning, and H. Haußecker . . . . . . . . 650
D. Engelmann, M. Stöhr, C. Garbe, and F. Hering . . . . . . . . . 648
S. Eichkorn, T. Münsterer, U. Lode and B. Jähne . . . . . . . . . . . . . . . . . . . . . 646
D. Uttenweiler and R. H. A. Fink . . . . . . . . 644
D. Schmundt and U. Schurr . . . . . . . . . . . . . . . . . . . . 642
U. Schimpf, H. Haußecker and B. Jähne . . . . . . . . . . . . . . . . . . . . . 640
B. Kümmerlen, S. Dauwe, D. Schmundt and U. Schurr . . . . . . . 638
P. Geißler and B. Jähne . . 636
P. Geißler and T. Scholz . . . . . . . . . . 634
Ch. Ridder, O. Munkelt and D. Hansel . . . . . . . . . . . . . . . . . . 630
ix
J.L. Barron, R.E. Mercer, D. Cheng, and P. Joe . . . . . . . . . . . . . . . . . 628
R. Frischholz . . . . . . . 626
R. Koch, M. Pollefeys and L. Von Gool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624
S. Karbacher and G. Häusler . . . . . . . 622
N. Stein and B. Minge . . . . . . . . 620
Torsten Scheuermann, Georg Wiora and Matthias Graf . 618
H.-G. Maas . . . . . . . . . . . . . . 616
H. Beyer . . . . . . . . . . . . . . . . . . . . . 614
T. Wagner, and P. Plankensteiner . . . . . . . . . . . . . . . . . 612
609 . . . . . . . . . . . 610
M. Wenig, C. Leue x
Contents
. . . . . . . . . 654
C. Leue, M. Wenig and U. Platt . . . . . . . . 656
C. Wolf, K. Meisenheimer, and H.-J. Roeser . . . . . . . . . . . . . . . . . 658
D. Uttenweiler and R. H. A. Fink . . . . . . . . . . . . . . . . 660
H. Bornfleth, P. Edelmann, and C. Cremer
667 Preface What this book is about
This book offers a fresh approach to computer vision. The whole vision process from image formation to measuring, recognition, or reacting is regarded as an integral process. Computer vision is understood as the host of techniques to acquire, process, analyze, and understand complex higher-dimensional data from our environment for scientific and technical exploration.
In this sense this book takes into account the interdisciplinary na- ture of computer vision with its links to virtually all natural sciences and attempts to bridge two important gaps. The first is between mod- ern physical sciences and the many novel techniques to acquire images. The second is between basic research and applications. When a reader with a background in one of the fields related to computer vision feels he has learned something from one of the many other facets of com- puter vision, the book will have fulfilled its purpose.
This book comprises three parts. The first part, Sensors and Imag-
ing, covers image formation and acquisition. The second part, Signal
Processing and Pattern Recognition, focuses on processing of the spatial
and spatiotemporal signals acquired by imaging sensors. The third part consists of an Application Gallery, which shows in a concise overview a wide range of application examples from both industry and science. This part illustrates how computer vision is integrated into a variety of systems and applications.
Computer Vision and Applications was designed as a concise edition
of the three-volume handbook:
Handbook of Computer Vision and Applications
edited by B. Jähne, H. Haußecker, and P. Geißler Vol 1: Sensors and Imaging; Vol 2: Signal Processing and Pattern Recognition; Vol 3: Systems and Applications Academic Press, 1999
Preface
xii It condenses the content of the handbook into one single volume and contains a selection of shortened versions of the most important contributions of the full edition. Although it cannot detail every single technique, this book still covers the entire spectrum of computer vision ranging from the imaging process to high-end algorithms and applica- tions. Students in particular can benefit from the concise overview of the field of computer vision. It is perfectly suited for sequential reading into the subject and it is complemented by the more detailed Handbook
of Computer Vision and Applications. The reader will find references
to the full edition of the handbook whenever applicable. In order to simplify notation we refer to supplementary information in the hand- book by the abbreviations [CVA1, Chapter N], [CVA2, Chapter N], and
th
[CVA3, Chapter N] for the N
chapter in the first, second and third volume, respectively. Similarly, direct references to individual sections in the handbook are given by [CVA1, Section N], [CVA2, Section N], and [CVA3, Section N] for section number N.
Prerequisites
It is assumed that the reader is familiar with elementary mathematical concepts commonly used in computer vision and in many other areas of natural sciences and technical disciplines. This includes the basics of set theory, matrix algebra, differential and integral equations, com- plex numbers, Fourier transform, probability, random variables, and graph theory. Wherever possible, mathematical topics are described intuitively. In this respect it is very helpful that complex mathematical relations can often be visualized intuitively by images. For a more for- mal treatment of the corresponding subject including proofs, suitable references are given.
How to use this book
The book has been designed to cover the different needs of its reader- ship. First, it is suitable for sequential reading. In this way the reader gets an up-to-date account of the state of computer vision. It is pre- sented in a way that makes it accessible for readers with different back- grounds. Second, the reader can look up specific topics of interest. The individual chapters are written in a self-consistent way with ex- tensive cross-referencing to other chapters of the book and external references. Additionally, a detailed glossary allows to easily access the most important topics independently of individual chapters. The CD that accompanies this book contains the complete text of the book in the Adobe Acrobat portable document file format (PDF). This format can be read on all major platforms. Free Acrobat™ Reader version 4.0
Preface
xiii for all major computing platforms is included on the CDs. The texts are hyperlinked in multiple ways. Thus the reader can collect the informa- tion of interest with ease. Third, the reader can delve more deeply into a subject with the material on the CDs. They contain additional refer- ence material, interactive software components, code examples, image material, and references to sources on the Internet. For more details see the readme file on the CDs.
Acknowledgments
Writing a book on computer vision with this breadth of topics is a major undertaking that can succeed only in a coordinated effort that involves many co-workers. Thus the editors would like to thank first all contrib- utors who were willing to participate in this effort. Their cooperation with the constrained time schedule made it possible that this concise edition of the Handbook of Computer Vision and Applications could be published in such a short period following the release of the handbook in May 1999. The editors are deeply grateful for the dedicated and pro- fessional work of the staff at AEON Verlag & Studio who did most of the editorial work. We also express our sincere thanks to Academic Press for the opportunity to write this book and for all professional advice.
Last but not least, we encourage the reader to send us any hints on errors, omissions, typing errors, or any other shortcomings of the book. Actual information about the book can be found at the editors homepage
Heidelberg, Germany, and Palo Alto, California Bernd Jähne, Horst Haußecker xiv
Contributors Prof. Dr. John L. Barron
Dept. of Computer Science, Middlesex College
The University of Western Ontario, London, Ontario, N6A 5B7, Canada
barron@csd.uwo.caHorst A. Beyer Imetric SA, Technopole, CH-2900 Porrentry, Switzerland imetric@dial.eunet.ch,
Dr. Harald Bornfleth Institut für Angewandte Physik, Universität Heidelberg Albert-Überle-Str. 3-5, D-69120Heidelberg, Germany Harald.Bornfleth@iwr.uni-heidelberg.de
David Cheng Dept. of Computer Science, Middlesex College
The University of Western Ontario, London, Ontario, N6A 5B7, Canada
cheng@csd.uwo.caProf. Dr. Christoph Cremer Institut für Angewandte Physik, Universität Heidelberg Albert-Überle-Str. 3-5, D-69120Heidelberg, Germany cremer@popeye.aphys2.uni-heidelberg.de
Tobias Dierig Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120Heidelberg Tobias Dierig@iwr.uni-heidelberg.de
Stefan Dauwe Botanisches Institut, Universität Heidelberg Im Neuenheimer Feld 360, D-69120 Heidelberg, Germany
Peter U. Edelmann Institut für Angewandte Physik, Universität Heidelberg Albert-Überle-Str. 3-5, D-69120Heidelberg, Germany edelmann@popeye.aphys2.uni-heidelberg.de
Sven Eichkorn
Max-Planck-Institut für Kernphysik, Abteilung Atmosphärenphysik
Contributors
xvi
Saupfercheckweg 1, D-69117 Heidelberg, Germany Sven.Eichkorn@mpi-hd.mpg.de Dirk Engelmann
Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120Heidelberg Dirk.Engelmann@iwr.uni-heidelberg.de
Prof. Dr. Rainer H. A. Fink
II. Physiologisches Institut, Universität Heidelberg Im Neuenheimer Feld 326, D-69120Heidelberg, Germany fink@novsrv1.pio1.uni-heidelberg.de
Dr. Robert Frischholz DCS AG, Wetterkreuz 19a, D-91058 Erlangen, Germany frz@dcs.de,
Christoph Garbe Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120Heidelberg Christoph.Garbe@iwr.uni-heidelberg.de
Dr. Peter Geißler ARRI, Abteilung TFE, Türkenstraße 95, D-80799 München pgeiss@hotmail.com
Dipl.-Ing. Robert Godding AICON GmbH, Celler Straße 32, D-38114 Braunschweig, Germany robert.godding@aicon.de,
Matthias Graf Institut für Kunststoffprüfung und Kunststoffkunde (IKP), Pfaffenwaldring 32, D-70569 Stuttgart, Germany graf@ikp.uni-stuttgart.de,Matthias.Graf@t-online.de
Hermann Gröning Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 360, D-69120 Heidelberg, Germany Hermann.Groening@iwr.uni-heidelberg.de
David Hansel FORWISS, Bayerisches Forschungszentrum für Wissensbasierte Systeme Forschungsgruppe Kognitive Systeme, Orleansstr. 34, 81667 München
Prof. Dr. Gerd Häusler Chair for Optics, Universität Erlangen-Nürnberg Staudtstraße 7/B2, D-91056 Erlangen, Germany haeusler@physik.uni-erlangen.de
Contributors
xvii
Dr. Horst Haußecker Xerox Palo Alto Research Center (PARC) 3333 Coyote Hill Road, Palo Alto, CA 94304 hhaussec@parc.xerox.com,
Dr. Frank Hering SAP AG, Neurottstraße 16, D-69190Walldorf, Germany frank.hering@sap.com
Dipl.-Inform. Thorsten Hermes Center for Computing Technology, Image Processing Department University of Bremen, P.O. Box 33 0440, D-28334 Bremen, Germany hermes@tzi.org,
Prof. Dr. Otthein Herzog Center for Computing Technology, Image Processing Department University of Bremen, P.O. Box 33 0440, D-28334 Bremen, Germany herzog@tzi.org,
Dr. Joachim Hornegger Lehrstuhl für Mustererkennung (Informatik 5)
Universität Erlangen-Nürnberg, Martensstraße 3, 91058 Erlangen, Germany
hornegger@informatik.uni-erlangen.deProf. Dr. Bernd Jähne Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120Heidelberg Bernd.Jaehne@iwr.uni-heidelberg.de
Dr. Paul Joe King City Radar Station, Atmospheric Environmental Services 4905 Dufferin St., Toronto, Ontario M3H 5T4, Canada joep@aestor.dots.doe.ca
Stefan Karbacher Chair for Optics, Universität Erlangen-Nürnberg Staudtstraße 7/B2, D-91056 Erlangen, Germany sbk@physik.uni-erlangen.de,
Prof. Dr.-Ing. Reinhard Koch Institut für Informatik und Praktische Mathematik
Christian-Albrechts-Universität Kiel, Olshausenstr. 40, D 24098 Kiel, Germany
rk@is.informatik.uni-kiel.deBernd Kümmerlen Botanisches Institut, Universität Heidelberg Im Neuenheimer Feld 360, D-69120 Heidelberg, Germany
Dr. Carsten Leue Institut für Umweltphysik, Universität Heidelberg Im Neuenheimer Feld 229, D-69120Heidelberg, Germany Carsten.Leue@iwr.uni-heidelberg.de
Ulrike Lode
Contributors
xviii
Im Neuenheimer Feld 229, D-69120Heidelberg, Germany
Prof. Dr.-Ing. Hans-Gerd Maas Institute for Photogrammetry and Remote Sensing Technical University Dresden, D-01062 Dresden, Germany maas@rcs.urz.tu-dresden.de
Prof. Dr.-Ing. Reinhard Malz Fachhochschule Esslingen, Fachbereich Informationstechnik Flandernstr. 101, D-73732 Esslingen reinhard.malz@fht-esslingen.de
Dr. Hanspeter A. Mallot Max-Planck-Institut für biologische Kybernetik Spemannstr. 38, 72076 Tübingen, Germany Hanspeter.Mallot@tuebingen.mpg.de
Prof. Robert E. Mercer Dept. of Computer Science, Middlesex College The University of Western Ontario, London, Ontario, N6A 5B7, Canada mercer@csd.uwo.ca
Dr. Anke Meyer-Bäse Dept. of Electrical Engineering and Computer Science University of Florida, 454 New Engineering Building 33, Center Drive PO Box 116130, Gainesville, FL 32611-6130, U.S. anke@alpha.ee.ufl.edu
Bernhard Minge
VITRONIC Dr.-Ing. Stein Bildverarbeitungssysteme GmbH Hasengartenstrasse 14a, D-65189 Wiesbaden, Germany bm@vitronic.de,
Dr. Olaf Munkelt FORWISS, Bayerisches Forschungszentrum für Wissensbasierte Systeme Forschungsgruppe Kognitive Systeme, Orleansstr. 34, 81667 München munkelt@forwiss.de,
Dr. Thomas Münsterer
VITRONIC Dr.-Ing. Stein Bildverarbeitungssysteme GmbH Hasengartenstr. 14a, D-65189 Wiesbaden, Germany Phone: +49-611-7152-38, tm@vitronic.de
Prof. Dr.-Ing. Heinrich Niemann Lehrstuhl für Mustererkennung (Informatik 5)
Universität Erlangen-Nürnberg, Martensstraße 3, 91058 Erlangen, Germany
niemann@informatik.uni-erlangen.deDr. Dietrich Paulus Lehrstuhl für Mustererkennung (Informatik 5)
Universität Erlangen-Nürnberg, Martensstraße 3, 91058 Erlangen, Germany
paulus@informatik.uni-erlangen.deContributors
xix
Dipl.-Math. Peter Plankensteiner Intego Plankensteiner Wagner Gbr Am Weichselgarten 7, D-91058 Erlangen ppl@intego.de
Prof. Dr. Ulrich Platt Institut für Umweltphysik, Universität Heidelberg Im Neuenheimer Feld 229, D-69120Heidelberg, Germany pl@uphys1.uphys.uni-heidelberg.de
Dr. Marc Pollefeys Katholieke Universiteit Leuven, ESAT-PSI/VISICS Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium Marc.Pollefeys@esat.kuleuven.ac.be
Christof Ridder FORWISS, Bayerisches Forschungszentrum für Wissensbasierte Systeme Forschungsgruppe Kognitive Systeme, Orleansstr. 34, 81667 München ridder@forwiss.de,
Dr. Torsten Scheuermann Fraunhofer USA, Headquarters 24 Frank Lloyd Wright Drive, Ann Arbor, MI 48106-0335, U.S. tscheuermann@fraunhofer.org,
Dr. Uwe Schimpf Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 360, D-69120 Heidelberg, Germany Uwe.Schimpf@iwr.uni-heidelberg.de
Dr. Dominik Schmundt Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 360, D-69120 Heidelberg, Germany Dominik.Schmundt@iwr.uni-heidelberg.de
Prof. Dr. Christoph Schnörr Dept. of Math. & Computer Science, University of Mannheim D-68131 Mannheim, Germany schnoerr@ti.uni-mannheim.de,
Dr. Thomas Scholz SAP AG, Neurottstraße 16, D-69190Walldorf, Germany thomas.scholz@sap.com
Dr. Ulrich Schurr Botanisches Institut, Universität Heidelberg Im Neuenheimer Feld 360, D-69120 Heidelberg, Germany uschurr@botanik1.bot.uni-heidelberg.de
Contributors
xx
Prof. Dr. Rudolf Schwarte Institut für Nachrichtenverarbeitung (INV) Universität-GH Siegen, Hölderlinstr. 3, D-57068 Siegen, Germany schwarte@nv.et-inf.uni-siegen.de
Prof. Dr. Peter Seitz Centre Suisse d’Electronique et de Microtechnique SA (CSEM) Badenerstrasse 569, CH-8048 Zurich, Switzerland peter.seitz@csem.ch,
Prof. Dr. Pierre Soille Silsoe Research Institute, Wrest Park Silsoe, Bedfordshire, MK45 4HS, United Kingdom Pierre.Soille@bbsrc.ac.uk,
Hagen Spies Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120Heidelberg Hagen.Spies@iwr.uni-heidelberg.de
Dr.-Ing. Norbert Stein
VITRONIC Dr.-Ing. Stein Bildverarbeitungssysteme GmbH Hasengartenstrasse 14a, D-65189 Wiesbaden, Germany st@vitronic.de,
Michael Stöhr Forschungsgruppe Bildverarbeitung, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120Heidelberg Michael.Stoehr@iwr.uni-heidelberg.de
Hamid R. Tizhoosh Universität Magdeburg (IPE) P.O. Box 4120, D-39016 Magdeburg, Germany tizhoosh@ipe.et.uni-magdeburg.de
Dr. Dietmar Uttenweiler
II. Physiologisches Institut, Universität Heidelberg Im Neuenheimer Feld 326, D-69120Heidelberg, Germany dietmar.uttenweiler@urz.uni-heidelberg.de
Prof. Dr. Luc Van Gool Katholieke Universiteit Leuven, ESAT-PSI/VISICS Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium luc.vangool@esat.kuleuven.ac.be
Dr. Thomas Wagner Intego Plankensteiner Wagner Gbr Am Weichselgarten 7, D-91058 Erlangen wag@intego.de
Contributors
xxi
Dr. Joachim Weickert Dept. of Math. & Computer Science, University of Mannheim D-68131 Mannheim, Germany Joachim.Weickert@ti.uni-mannheim.de
Mark O. Wenig Institut für Umweltphysik, Universität Heidelberg Im Neuenheimer Feld 229, D-69120Heidelberg, Germany Mark.Wenig@iwr.uni-heidelberg.de
Georg Wiora DaimlerChrysler AG, Research and Development Wilhelm-Runge-Str. 11, D-89081 Ulm, Germany georg.wiora@DaimlerChrysler.com
Dr. Christian Wolf Max-Planck Institut für Astronomie Königstuhl 17, D-69117 Heidelberg cwolf@mpia-hd.mpg.de xxii
1 Introduction
Interdisziplinäres Zentrum für Wissenschaftliches Rechnen (IWR) Universität Heidelberg,Germany . . . . . . . . . . . . . . . . . . .
1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 . . . . . . . . . . . . . . .
3 . . . . . . . . . . . . . .
4 . . . . . . . . . . . . . . .
5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
1.1 Components of a vision system
Computer vision is a complex subject. As such it is helpful to divide it into its various components or function modules. On this level, it is also much easier to compare a technical system with a biological system. In this sense, the basic common functionality of biological and machine vision includes the following components (see also Table
Radiation source. If no radiation is emitted from the scene or the ob- ject of interest, nothing can be observed or processed. Thus appro- priate illumination is necessary for objects that are themselves not radiant.
Camera. The “camera” collects the radiation received from the object in such a way that the radiation’s origins can be pinpointed. In the simplest case this is just an optical lens. But it could also be a completely different system, for example, an imaging optical spec- trometer, an x-ray tomograph, or a microwave dish. Sensor. The sensor converts the received radiative flux density into a suitable signal for further processing. For an imaging system nor- mally a 2-D array of sensors is required to capture the spatial dis- tribution of the radiation. With an appropriate scanning system in some cases a single sensor or a row of sensors could be sufficient.
2
1 Introduction
Table 1.1: Function modules of human and machine vision
Task Human vision Machine vision
Visualization Passive,mainly by re- Passive and active (controlled il-
flection of light from lumination) using electromagnetic, opaque surfaces particulate,and acoustic radiationImage Refractive optical sys- Various systems formation tem
Control of Muscle-controlled pupil Motorized apertures,filter wheels,
irradiance tunable filtersFocusing Muscle-controlled Autofocus systems based on vari-
change of focal length ous principles of distance measure- mentsIrradiance Logarithmic sensitivity Linear sensitivity,quantization be-
resolution tween 8- and 16-bits; logarithmic
sensitivityTracking Highly mobile eyeball Scanner and robot-mounted cam-
erasProcessing Hierarchically Serial processing still dominant;
and analysis organized massively parallel processing not in general
parallel processing useProcessing unit. It processes the incoming, generally higher-dimen- sional data, extracting suitable features that can be used to measure object properties and categorize them into classes. Another impor- tant component is a memory system to collect and store knowl- edge about the scene, including mechanisms to delete unimportant things. Actors. Actors react to the result of the visual observation. They be- come an integral part of the vision system when the vision system is actively responding to the observation by, for example, tracking an object of interest or by using a vision-guided navigation (active
vision, perception action cycle).
1.2 Imaging systems
Imaging systems cover all processes involved in the formation of an image from objects and the sensors that convert radiation into elec- tric signals, and further into digital signals that can be processed by a computer. Generally the goal is to attain a signal from an object in such a form that we know where it is (geometry), and what it is or what properties it has.
1.3 Signal processing for computer vision
3 Radiance Irradiance
Property Electric Digital Object signal image ADC
Imaging Photo- radiation sampling system sensor
G
s(x) interaction l(x) E(x) g(x) mn
Figure 1.1: Chain of steps linking an object property to the signal measured by
an imaging system.It is important to note that the type of answer we receive from these two implicit questions depends on the purpose of the vision system. The answer could be of either a qualitative or a quantitative nature. For some applications it could be sufficient to obtain a qualitative an- swer like “there is a car on the left coming towards you.” The “what” and “where” questions can thus cover the entire range from “there is something,” a specification of the object in the form of a class, to a de- tailed quantitative description of various properties of the objects of interest.
The relation that links the object property to the signal measured by an imaging system is a complex chain of processes (Fig.
Interaction
of the radiation with the object (possibly using an appropriate illumi- nation system) causes the object to emit radiation. A portion (usually only a very small part) of the emitted radiative energy is collected by the optical system and perceived as an irradiance (radiative energy/area). A sensor (or rather an array of sensors) converts the received radiation into an electrical signal that is subsequently sampled and digitized to form a digital image as an array of digital numbers.
Only direct imaging systems provide a direct point-to-point corre- spondence between points of the objects in the 3-D world and at the image plane. Indirect imaging systems also give a spatially distributed irradiance but with no such one-to-one relation. Generation of an im- age requires reconstruction of the object from the perceived irradiance. Examples of such imaging techniques include radar imaging, various techniques for spectral imaging, acoustic imaging, tomographic imag- ing, and magnetic resonance imaging.
1.3 Signal processing for computer vision
One-dimensional linear signal processing and system theory is a stan- dard topic in electrical engineering and is covered by many standard textbooks (e.g.,
There is a clear trend that the classical signal
processing community is moving into multidimensional signals, as in- dicated, for example, by the new annual international IEEE conference on image processing (ICIP). This can also be seen from some recently published handbooks on this subject. The digital signal processing handbook by Madisetti and Williams
includes several chapters that
4
1 Introduction
deal with image processing. Likewise the transforms and applications handbook by Poularikas
is not restricted to 1-D transforms.
There are, however, only a few monographs that treat signal pro- cessing specifically for computer vision and image processing. The monograph by Lim
deals with 2-D signal and image processing and
tries to transfer the classical techniques for the analysis of time series to 2-D spatial data. Granlund and Knutsson
were the first to publish
a monograph on signal processing for computer vision and elaborate on a number of novel ideas such as tensorial image processing and nor- malized convolution that did not have their origin in classical signal processing.
Time series are 1-D, signals in computer vision are of higher di- mension. They are not restricted to digital images, that is, 2-D spatial signals (Chapter
Volumetric sampling, image sequences, and hyper-
spectral imaging all result in 3-D signals, a combination of any of these
techniques in even higher-dimensional signals.How much more complex does signal processing become with in- creasing dimension? First, there is the explosion in the number of data
3
points. Already a medium resolution volumetric image with 512 vox- els requires 128 MB if one voxel carries just one byte. Storage of even higher-dimensional data at comparable resolution is thus beyond the capabilities of today’s computers.
Higher dimensional signals pose another problem. While we do not have difficulty in grasping 2-D data, it is already significantly more de- manding to visualize 3-D data because the human visual system is built only to see surfaces in 3-D but not volumetric 3-D data. The more di- mensions are processed, the more important it is that computer graph- ics and computer vision move closer together.
The elementary framework for lowlevel signal processing for com- puter vision is worked out in Chapters
Of central importance
are neighborhood operations (Chapter
including fast algorithms for
local averaging (Section
1.4 Pattern recognition for computer vision
The basic goal of signal processing in computer vision is the extraction of “suitable features” for subsequent processing to recognize and clas- sify objects. But what is a suitable feature? This is still less well defined than in other applications of signal processing. Certainly a mathemat- ically well-defined description of local structure as discussed in Sec- tion
is an important basis. As signals processed in computer vision
come from dynamical 3-D scenes, important features also include mo-
tion (Chapter and various techniques to infer the depth in scenes
1.5 Performance evaluation of algorithms
5 including stereo (Section
shape from shading and photometric
stereo, and depth from focus (Section
There is little doubt that nonlinear techniques are crucial for fea- ture extraction in computer vision. However, compared to linear filter techniques, these techniques are still in their infancy. There is also no single nonlinear technique but there are a host of such techniques often specifically adapted to a certain purpose
]. In this volume, we
give an overview of the various classes of nonlinear filter techniques (Section
and focus on a first-order tensor representation of of non-
linear filters by combination of linear convolution and nonlinear point operations (Chapter