3D BUILDING RECONSTRUCTION BY MULTIVIEW IMAGES AND THE INTEGRATED APPLICATION WITH AUGMENTED REALITY
Jin-Tsong Hwang , Ting-Chen Chu
Dept. of Real Estate and Built Environment, National Taipei University, Taiwan - E-mail: jthwangmail.ntpu.edu.tw
; applechu0417gmail.com
Commission I, WG IVb
KEY WORDS: Point cloud, Augment Reality, Unmanned aerial vehicle, Structure from Motion ABSTRACT:
This study presents an approach wherein photographs with a high degree of overlap are clicked using a digital camera and used to generate three-dimensional 3D point clouds via feature point extraction and matching. To reconstruct a building model, an
unmanned aerial vehicle UAV is used to click photographs from vertical shooting angles above the building. Multiview images are taken from the ground to eliminate the shielding effect on UAV images caused by trees. Point clouds from the UAV and multiview
images are generated via Pix4Dmapper. By merging two sets of point clouds via tie points, the complete building model is reconstructed. The 3D models are reconstructed using AutoCAD 2016 to generate vectors from the point clouds; SketchUp Make
2016 is used to rebuild a complete building model with textures. To apply 3D building models in urban planning and design, a modern approach is to rebuild the digital models; however, replacing the landscape design and building distribution in real time is
difficult as the frequency of building replacement increases. One potential solution to these problems is augmented reality AR. Using Unity3D and Vuforia to design and implement the smartphone application service, a markerless AR of the building model can
be built. This study is aimed at providing technical and design skills related to urban planning, urban designing, and building information retrieval using AR.
Corresponding author
1. INTRODUCTION
A conceptual model to organize the various tools for visualization in urban planning was designed by Al-Kodmany
2002. According to this model, non-computer-based tools can be classified as 1 a pen and paper, 2 paper maps, 3
photographs, and 4 three-dimensional 3D physical models. The corresponding computer-based tools are 1 an electronic
pen and paper; 2 geographical information systems GIS, computer-aided mapping CAM, and mapping information
management systems MIMS; 3 image editing programs, digital photographs, and digital video; and 4 3D digital
models, virtual reality, and urban simulators Al-Kodmany, 2002. Therefore, augmented reality AR tools belong most
strongly to the fourth category. Digital building models are necessary for urban design and planning, 3D city modeling, and
several other applications. Aerial imagery has traditionally been a data source for digital building model generation, and various
aerial imagery approaches have been investigated and developed. Aerial imagery shooting by rotorcraft-based
unmanned aerial vehicles UAVs is suitable for reconstructing 3D building models because photographs can be clicked at
fixed camera stations and multiview images can be obtained. In this study, a rotorcraft-based UAV was used for data
acquisition. The flight plan entailed vertical shooting on the building to generate a 3D building model. Because vertical
shooting suffers from shading by objects, multiview images from the ground must be included to remove the shielding
effect. Classical photogrammetry is rarely adopted to obtain high-
resolution and high-accuracy products through UAV images. New computer-vision algorithms and techniques enable
superior output generation Lee et al., 2014. Muneza et al. 2015 used UAV imagery to produce a high-quality orthophoto
to update geospatial data. A DJI Phantom 2 Vision Plus quadcopter was used to collect 954 images at a flying height of
50 m. Using appropriate photogrammetric software, an orthophoto with a 3.3-cm ground sample distance GSD
covering 0.095 km² was produced. With appropriate ground control points, an absolute positional accuracy of a 7.9 cm root-
mean-square error RMSE was achieved.
Many software products are based on image-based modeling, such as Photosynth, Speeded Up Robust Features SURF,
VisualSFM, and Pix4Dmapper. These products reconstruct the spatial geometry of the shooting subjects by searching for
feature points to match and stitch images, restore camera poses, and inquire about the coordinates of the shooting subject.
The traditional approach for applying a 3D building model in urban planning and design involves one person operating a
computer to display the digital building on the planned location after designing and space planning. Thus, the location and the
view angle are controlled unilaterally by the operator on the
This contribution has been peer-reviewed. doi:10.5194isprs-archives-XLI-B1-1235-2016
1235
scene of the display; other audiences can only watch the result passively. A modern approach is to rebuild the digital models;
however, it is difficult to replace the landscape design and the building distribution in real time as the frequency of building
replacement increases. AR is a potential solution to these problems.
The conceptual basis of AR is the integration of the real world with the virtual world. Using computer graphics techniques, an
object is generated with necessary information and then added into the real environment. By using computer vision, AR can
recognize markers, such as simple black-and-white 2D-markers or real environment surroundings Ruan and Jeong, 2012; Lin et
al., 2009. These tracking methods can be divided into two different classes: marker-based and markerless. In the marker-
based approach, different information can be displayed on a variety of surfaces depending on the purpose of the system Gao
et al., 2014. Meanwhile, markerless tracking based on image processing uses natural features in the images to evaluate the
camera pose. AR can improve the recognition tools for the real world and thus enable efficient interactions between humans
and computers Kim, Young-geun, and Won-jung Kim, 2014.
Alexandro Simonetti Ibañez 2013 describes what is understood by AR and the different varieties of AR applications
and then discusses the software development kit SDK’s features, architecture, and elements. This article describes the
basis of the detection process realized by the Vuforia library
2. METHODOLOGY 2.1
