smr_defense.ppt 3624KB Jun 23 2011 12:31:06 PM
Real-time Acquisition and
Rendering of Large 3D
Models
Szymon Rusinkiewicz
Computer Graphics Pipeline
Shape
Shape
Rendering
3D Scanning
Motion
Motion
Lighting
and
Reflectance
Reflectance
• Human time = expensive
• Sensors = cheap
– Computer graphics increasingly relies on
measurements of the real world
3D Scanning Applications
• Computer
graphics
• Product
inspection
• Robot
navigation
• As-built
floorplans
• Product design
• Archaeology
• Clothes fitting
• Art history
The Digital Michelangelo
Project
• Push state of the art in range
scanning and demonstrate
applications in art and art history
Working in
the museum
Scanning
geometry
Scanning
color
Traditional Range Scanning
Pipeline
• High-quality, robust pipeline for
producing 3D models:
– Scan object with laser triangulation
scanner: many views from different angles
– Align pieces into single coordinate frame:
initial manual alignment, refined with ICP
– Merge overlapping regions: compute
“average” surface using VRIP [Curless &
Levoy 96]
– Display resulting model
3D Scan of David:
Statistics
• Over 5 meters tall
• 1/4 mm resolution
• 22 people
• 30 nights of scanning
• Efficiency max : min = 8 : 1
– Needed view planning
• Weight of gantry: 800 kg
• Putting model together:
1000+ man-hours and
counting
New 3D Scanning Pipeline
• Need for a fast, inexpensive,
easy-to-use 3D scanning system
• Wave a (small, rigid)
object by hand in front of
the scanner
• Automatically align data
as
it is acquired
• Let user see partial
Model Acquisition
model as Real-Time
it
Real-Time 3D Model
Acquisition
• Prototype real-time model
acquisition system
– 3D scanning of moving objects
– Fast alignment
– Real-time merging and display
Applications of Easy-to-Use
3D Model Acquisition
• Advertising
• More capabilities in Photoshop
• Movie sets
• Augmented reality
• User interfaces
3D Scanning Technologies
• Contact-based: touch probes
• Passive: shape from stereo, motion,
shading
• Active: time-of-flight, defocus,
photometric stereo, triangulation
• Triangulation systems are
inexpensive, robust, and flexible
– Take advantage of trends in DLP
projectors
Laser Triangulation
Object
Laser
Camera
Camera
• Project laser stripe onto object
Laser Triangulation
Object
Laser
(x,y)
Camera
Camera
• Depth from ray-plane triangulation
Triangulation
• Faster acquisition: project
multiple stripes
• Correspondence problem: which
stripe
is which?
Triangulation
Multi-stripe
Multi-frame
Single-stripe
Slow, robust
Single-frame
Fast, fragile
Time-Coded Light Patterns
• Assign each stripe a unique
illumination code
over time [Posdamer 82]
Time
Time
Space
Gray-Code Patterns
• To minimize effects of quantization error:
each point may be a boundary only once
Time
Space
Structured-Light
Assumptions
• Structured-light systems make certain
assumptions about the scene:
• Spatial continuity assumption:
– Assume scene is one object
– Project a grid, pattern of dots, etc.
• Temporal continuity assumption:
– Assume scene is static
– Assign stripes a code over time
Codes for Moving Scenes
• We make a different assumption:
– Object may move
– Velocity low enough to permit
tracking
– “Spatio-temporal” continuity
Codes for Moving Scenes
• Code stripe boundaries
instead of stripes
• Perform frame-to-frame
tracking of corresponding
boundaries
– Propagate illumination
history
Illumination
Illumination history
history = (WB),(BW),(WB)
(WB),(BW),(WB)
[Hall-Holt & Rusinkiewicz, ICCV
Code
New Scanning Pipeline
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
Designing a Code
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Biggest problem is ghosts – WW or
BB “boundaries” that can’t be
seen directly
Designing a Code
• Design a code to make tracking
possible:
– Do not allow two spatially adjacent
ghosts
– Do not allow two temporally adjacent
ghosts
t
Designing a Code
• Graph (for 4 frames):
0000
1101
– Nodes: stripes (over time)
– Edges: boundaries (over time)
1010 0111
0101
1011
1111
0010
0001
1100
1000
0110
0100 1001
Time
1110
Space
0011
Designing a Code
• Graph (for 4 frames):
0000
1101
– Nodes: stripes (over time)
– Edges: boundaries (over time)
1010 0111
Boundary visible at even times 0101
1111
0010
0001
1100
1000
Boundary visible at odd times
1011
0110
0100 1001
• Path with alternating colors:
1110
55 edges in graph
maximal-length traversal has 110 boundaries (111
stripes)
0011
Image Capture
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Standard video camera: fields at
60 Hz
• Genlock camera to projector
Finding Boundaries
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Standard edge detection problem
• Current solution: find minima and
maxima of intensity, boundary is
between them
Matching Stripe Boundaries
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Even if number of ghosts is
minimized, matching is not easy
?
Matching Stripe Boundaries
• Resolve ambiguity by constraining
maximum stripe velocity
• Could accommodate higher speeds
by estimating velocities
• Could take advantage of methods
in
tracking literature (e.g., Kalman
filters)
Decoding Boundaries
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Propagate illumination history
• Table lookup based on illumination
history and position in four-frame
sequence
– Once a stripe has been tracked for at least
four frames,
it contributes useful data on every
subsequent frame
Computing 3D Position
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
• Ray-plane intersection
• Requires calibration of:
– Camera, projector intrinsics
– Relative position and orientation
Compute
Compute
Range
Results
Video
frames
Stripe
boundaries
unknow
n
known
ghosts
Results
• Single range image of moving
object
Top View
Front View
Gray codes, no tracking
Top View
Front View
Boundary codes and tracking
Aligning 3D Data
• This range scanner can be used
for any moving objects
• For rigid objects, range images
can be aligned to each other as
object moves
Aligning 3D Data
• If correct correspondences are
known,
it is possible to find correct
relative rotation/translation
Aligning 3D Data
• How to find corresponding points?
• Previous systems based on user
input,
feature matching, surface
signatures, etc.
Aligning 3D Data
• Alternative: assume closest points
correspond to each other, compute
the best transform…
Aligning 3D Data
• … and iterate to find alignment
– Iterated Closest Points (ICP) [Besl &
McKay 92]
• Converges if starting position
“close enough“
ICP Variants
• Classic ICP algorithm not real-time
• To improve speed: examine stages of ICP and evaluate proposed variants
1.
2.
3.
4.
5.
6.
Selecting source points (from one or both mesh
Matching to points in the other mesh
Weighting the correspondences
Rejecting certain (outlier) point pairs
Assigning an error metric to the current transfo
transf
Minimizing the error metric
[Rusinkiewicz & Levoy, 3DIM 2001]
ICP Variant –
Point-to-Plane Error Metric
• Using point-to-plane distance
instead of point-to-point lets flat
regions slide along each other
more easily [Chen & Medioni 91]
Finding Corresponding
Points
• Finding closest point is most expensive
stage of ICP
– Brute force search – O(n)
– Spatial data structure (e.g., k-d tree) –
O(log n)
– Voxel grid – O(1), but large constant, slow
preprocessing
Finding Corresponding
Points
• For range images, simply project point
[Blais 95]
– Constant-time, fast
– Does not require precomputing a spatial data
structure
High-Speed ICP Algorithm
• ICP algorithm with projectionbased correspondences, point-toplane matching
can align meshes in a few tens of
ms.
(cf. over 1 sec. with closest-point)
Anchor Scans
• Alignment of consecutive scans
leads to accumulation of ICP
errors
• Alternative: align all scans to an
“anchor” scan, only switch anchor
when overlap low
• Given anchor scans, restart after
failed ICP becomes easier
Merging and Rendering
• Goal: visualize the model well enough
to be able to see holes
• Cannot display all the scanned data –
accumulates linearly with time
• Standard high-quality merging
methods:
processing time ~ 1 minute per scan
Merging and Rendering
• Real-time incremental merging
and rendering:
– Quantize samples to a 3D grid
– Maintain average normal of all points
at a grid cell
– Point (splat) rendering
– Can be made hierarchical to conserve
memory
Photograph
Real-time Scanning Demo
Postprocessing
• Goal of real-time display is to let
user evaluate coverage, fill holes
– Quality/speed tradeoff
• Offline postprocessing for highquality models
Merged Result
Photograph
Aligned scans
Merged
Future Work
• Technological improvements:
–
–
–
Use full resolution of projector
Higher-resolution cameras
Ideas from design of single-stripe 3D scanners
• Pipeline improvements:
– Better detection of failed alignment
– Better handling of object texture – combine
with stereo?
– Global registration to eliminate drift
– More sophisticated merging
– Improve user interaction during scanning
Future Work
• Faster scanning
– Better stripe boundary matching
– Multiple cameras, projectors
– High-speed cameras
• Application in different contexts
– Small, hand-held
– Cart- or shoulder-mounted for digitizing
rooms
– Infrared for imperceptibility
Rendering of Large Models
• Range scanners increasingly capable of
producing very large models
– DMich models are 100 million to 1 billion samples
• Challenge: how to allow viewing in real time
– Fast startup, progressive loading
• Traditional answer: triangle meshes,
simplification, hardware-accelerated
rendering
– Impractical for such large models
• Alternative: revisit basic data structure
QSplat [Rusinkiewicz & Levoy, SIGGRAPH 00]
QSplat
• Key observation: a single
bounding sphere hierarchy can be
used for
– Hierarchical frustum and backface
culling
– Level of detail control
– Splat rendering [Westover 89]
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
14 bits
6 bytes
2 bits
Color
(Optional)
16 bits
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
14 bits
2 bits
• Position and radius encoded
relative to parent node
Color
(Optional)
16 bits
Center Offset
– Hierarchical coding vs. delta
coding along a path for vertex
positions
Radius Ratio
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
Uncompressed
14 bits
2 bits
Color
(Optional)
16 bits
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
Delta Coding
[Deering 96]
14 bits
2 bits
Color
(Optional)
16 bits
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
Hierarchical
Coding
3 bits
14 bits
2 bits
Color
(Optional)
16 bits
QSplat Rendering Algorithm
• Traverse hierarchy recursively
Hierarchical frustum
Point rendering
Hierarchical frustum
backface culling
if (node not visible)
Skip this branch
else if (leaf node)
Level of detail
control
Draw a splat
else if (size on screen <
threshold)
Draw a splat
else
Traverse children
Adjusted to maintai
mainta
desired frame rate
Demo – St. Matthew
• 3D scan of 2.7 meter
statue at 0.25 mm
• 102,868,637 points
• File size: 644 MB
• Preprocessing time:
1 hour
Future Work
• Splats as primitive
– Unify rendering of meshes, volumes, point
clouds
– Compatible with shading after rasterization
– Hybrid point/polygon systems
• High-level visibility / LOD frameworks
– Store different kinds of data at each node:
alpha, BRDF, scattering function, etc.
– Potentially could be used to unify imagebased-rendering (IBR) techniques
Contributions
• Real-time 3D model acquisition system
–
–
–
–
Video-rate 3D scanner for moving objects
Analysis of ICP variants; real-time algorithm
Real-time merging and rendering
Allows user to see model and fill holes
• QSplat: interactive rendering of large 3D
meshes
– Single data structure used for visibility culling,
level-of-detail control, point rendering,
compression
– Extension to network streaming [I3D 2001]
Acknowledgments
• Olaf Hall-Holt
• Lucas Pereira
• The Original DMich Gang: Dave Koller, Sean
Anderson, James Davis, Kari Pulli, Matt Ginzton, Jon
Shade
• DMich, the next generation: Gary King, Steve
Marschner
• Graphics lab
• Advisor: Marc Levoy
• Committee: Pat Hanrahan, Leo Guibas, Mark
Horowitz, Bernd Girod
• Family, friends
• Sponsors: NSF, Interval, Honda, Sony, Intel
Rendering of Large 3D
Models
Szymon Rusinkiewicz
Computer Graphics Pipeline
Shape
Shape
Rendering
3D Scanning
Motion
Motion
Lighting
and
Reflectance
Reflectance
• Human time = expensive
• Sensors = cheap
– Computer graphics increasingly relies on
measurements of the real world
3D Scanning Applications
• Computer
graphics
• Product
inspection
• Robot
navigation
• As-built
floorplans
• Product design
• Archaeology
• Clothes fitting
• Art history
The Digital Michelangelo
Project
• Push state of the art in range
scanning and demonstrate
applications in art and art history
Working in
the museum
Scanning
geometry
Scanning
color
Traditional Range Scanning
Pipeline
• High-quality, robust pipeline for
producing 3D models:
– Scan object with laser triangulation
scanner: many views from different angles
– Align pieces into single coordinate frame:
initial manual alignment, refined with ICP
– Merge overlapping regions: compute
“average” surface using VRIP [Curless &
Levoy 96]
– Display resulting model
3D Scan of David:
Statistics
• Over 5 meters tall
• 1/4 mm resolution
• 22 people
• 30 nights of scanning
• Efficiency max : min = 8 : 1
– Needed view planning
• Weight of gantry: 800 kg
• Putting model together:
1000+ man-hours and
counting
New 3D Scanning Pipeline
• Need for a fast, inexpensive,
easy-to-use 3D scanning system
• Wave a (small, rigid)
object by hand in front of
the scanner
• Automatically align data
as
it is acquired
• Let user see partial
Model Acquisition
model as Real-Time
it
Real-Time 3D Model
Acquisition
• Prototype real-time model
acquisition system
– 3D scanning of moving objects
– Fast alignment
– Real-time merging and display
Applications of Easy-to-Use
3D Model Acquisition
• Advertising
• More capabilities in Photoshop
• Movie sets
• Augmented reality
• User interfaces
3D Scanning Technologies
• Contact-based: touch probes
• Passive: shape from stereo, motion,
shading
• Active: time-of-flight, defocus,
photometric stereo, triangulation
• Triangulation systems are
inexpensive, robust, and flexible
– Take advantage of trends in DLP
projectors
Laser Triangulation
Object
Laser
Camera
Camera
• Project laser stripe onto object
Laser Triangulation
Object
Laser
(x,y)
Camera
Camera
• Depth from ray-plane triangulation
Triangulation
• Faster acquisition: project
multiple stripes
• Correspondence problem: which
stripe
is which?
Triangulation
Multi-stripe
Multi-frame
Single-stripe
Slow, robust
Single-frame
Fast, fragile
Time-Coded Light Patterns
• Assign each stripe a unique
illumination code
over time [Posdamer 82]
Time
Time
Space
Gray-Code Patterns
• To minimize effects of quantization error:
each point may be a boundary only once
Time
Space
Structured-Light
Assumptions
• Structured-light systems make certain
assumptions about the scene:
• Spatial continuity assumption:
– Assume scene is one object
– Project a grid, pattern of dots, etc.
• Temporal continuity assumption:
– Assume scene is static
– Assign stripes a code over time
Codes for Moving Scenes
• We make a different assumption:
– Object may move
– Velocity low enough to permit
tracking
– “Spatio-temporal” continuity
Codes for Moving Scenes
• Code stripe boundaries
instead of stripes
• Perform frame-to-frame
tracking of corresponding
boundaries
– Propagate illumination
history
Illumination
Illumination history
history = (WB),(BW),(WB)
(WB),(BW),(WB)
[Hall-Holt & Rusinkiewicz, ICCV
Code
New Scanning Pipeline
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
Designing a Code
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Biggest problem is ghosts – WW or
BB “boundaries” that can’t be
seen directly
Designing a Code
• Design a code to make tracking
possible:
– Do not allow two spatially adjacent
ghosts
– Do not allow two temporally adjacent
ghosts
t
Designing a Code
• Graph (for 4 frames):
0000
1101
– Nodes: stripes (over time)
– Edges: boundaries (over time)
1010 0111
0101
1011
1111
0010
0001
1100
1000
0110
0100 1001
Time
1110
Space
0011
Designing a Code
• Graph (for 4 frames):
0000
1101
– Nodes: stripes (over time)
– Edges: boundaries (over time)
1010 0111
Boundary visible at even times 0101
1111
0010
0001
1100
1000
Boundary visible at odd times
1011
0110
0100 1001
• Path with alternating colors:
1110
55 edges in graph
maximal-length traversal has 110 boundaries (111
stripes)
0011
Image Capture
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Standard video camera: fields at
60 Hz
• Genlock camera to projector
Finding Boundaries
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Standard edge detection problem
• Current solution: find minima and
maxima of intensity, boundary is
between them
Matching Stripe Boundaries
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Even if number of ghosts is
minimized, matching is not easy
?
Matching Stripe Boundaries
• Resolve ambiguity by constraining
maximum stripe velocity
• Could accommodate higher speeds
by estimating velocities
• Could take advantage of methods
in
tracking literature (e.g., Kalman
filters)
Decoding Boundaries
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
Compute
Compute
Range
• Propagate illumination history
• Table lookup based on illumination
history and position in four-frame
sequence
– Once a stripe has been tracked for at least
four frames,
it contributes useful data on every
subsequent frame
Computing 3D Position
Project
Project
Code
Code
Capture
Capture
Images
Find
Boundaries
Boundaries
Match
Boundaries
Boundaries
Decode
Decode
• Ray-plane intersection
• Requires calibration of:
– Camera, projector intrinsics
– Relative position and orientation
Compute
Compute
Range
Results
Video
frames
Stripe
boundaries
unknow
n
known
ghosts
Results
• Single range image of moving
object
Top View
Front View
Gray codes, no tracking
Top View
Front View
Boundary codes and tracking
Aligning 3D Data
• This range scanner can be used
for any moving objects
• For rigid objects, range images
can be aligned to each other as
object moves
Aligning 3D Data
• If correct correspondences are
known,
it is possible to find correct
relative rotation/translation
Aligning 3D Data
• How to find corresponding points?
• Previous systems based on user
input,
feature matching, surface
signatures, etc.
Aligning 3D Data
• Alternative: assume closest points
correspond to each other, compute
the best transform…
Aligning 3D Data
• … and iterate to find alignment
– Iterated Closest Points (ICP) [Besl &
McKay 92]
• Converges if starting position
“close enough“
ICP Variants
• Classic ICP algorithm not real-time
• To improve speed: examine stages of ICP and evaluate proposed variants
1.
2.
3.
4.
5.
6.
Selecting source points (from one or both mesh
Matching to points in the other mesh
Weighting the correspondences
Rejecting certain (outlier) point pairs
Assigning an error metric to the current transfo
transf
Minimizing the error metric
[Rusinkiewicz & Levoy, 3DIM 2001]
ICP Variant –
Point-to-Plane Error Metric
• Using point-to-plane distance
instead of point-to-point lets flat
regions slide along each other
more easily [Chen & Medioni 91]
Finding Corresponding
Points
• Finding closest point is most expensive
stage of ICP
– Brute force search – O(n)
– Spatial data structure (e.g., k-d tree) –
O(log n)
– Voxel grid – O(1), but large constant, slow
preprocessing
Finding Corresponding
Points
• For range images, simply project point
[Blais 95]
– Constant-time, fast
– Does not require precomputing a spatial data
structure
High-Speed ICP Algorithm
• ICP algorithm with projectionbased correspondences, point-toplane matching
can align meshes in a few tens of
ms.
(cf. over 1 sec. with closest-point)
Anchor Scans
• Alignment of consecutive scans
leads to accumulation of ICP
errors
• Alternative: align all scans to an
“anchor” scan, only switch anchor
when overlap low
• Given anchor scans, restart after
failed ICP becomes easier
Merging and Rendering
• Goal: visualize the model well enough
to be able to see holes
• Cannot display all the scanned data –
accumulates linearly with time
• Standard high-quality merging
methods:
processing time ~ 1 minute per scan
Merging and Rendering
• Real-time incremental merging
and rendering:
– Quantize samples to a 3D grid
– Maintain average normal of all points
at a grid cell
– Point (splat) rendering
– Can be made hierarchical to conserve
memory
Photograph
Real-time Scanning Demo
Postprocessing
• Goal of real-time display is to let
user evaluate coverage, fill holes
– Quality/speed tradeoff
• Offline postprocessing for highquality models
Merged Result
Photograph
Aligned scans
Merged
Future Work
• Technological improvements:
–
–
–
Use full resolution of projector
Higher-resolution cameras
Ideas from design of single-stripe 3D scanners
• Pipeline improvements:
– Better detection of failed alignment
– Better handling of object texture – combine
with stereo?
– Global registration to eliminate drift
– More sophisticated merging
– Improve user interaction during scanning
Future Work
• Faster scanning
– Better stripe boundary matching
– Multiple cameras, projectors
– High-speed cameras
• Application in different contexts
– Small, hand-held
– Cart- or shoulder-mounted for digitizing
rooms
– Infrared for imperceptibility
Rendering of Large Models
• Range scanners increasingly capable of
producing very large models
– DMich models are 100 million to 1 billion samples
• Challenge: how to allow viewing in real time
– Fast startup, progressive loading
• Traditional answer: triangle meshes,
simplification, hardware-accelerated
rendering
– Impractical for such large models
• Alternative: revisit basic data structure
QSplat [Rusinkiewicz & Levoy, SIGGRAPH 00]
QSplat
• Key observation: a single
bounding sphere hierarchy can be
used for
– Hierarchical frustum and backface
culling
– Level of detail control
– Splat rendering [Westover 89]
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
14 bits
6 bytes
2 bits
Color
(Optional)
16 bits
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
14 bits
2 bits
• Position and radius encoded
relative to parent node
Color
(Optional)
16 bits
Center Offset
– Hierarchical coding vs. delta
coding along a path for vertex
positions
Radius Ratio
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
Uncompressed
14 bits
2 bits
Color
(Optional)
16 bits
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
3 bits
Delta Coding
[Deering 96]
14 bits
2 bits
Color
(Optional)
16 bits
QSplat Node Structure
Position
Width of
Tree
and Structure Normal Cone of
Radius
Normals
13 bits
Hierarchical
Coding
3 bits
14 bits
2 bits
Color
(Optional)
16 bits
QSplat Rendering Algorithm
• Traverse hierarchy recursively
Hierarchical frustum
Point rendering
Hierarchical frustum
backface culling
if (node not visible)
Skip this branch
else if (leaf node)
Level of detail
control
Draw a splat
else if (size on screen <
threshold)
Draw a splat
else
Traverse children
Adjusted to maintai
mainta
desired frame rate
Demo – St. Matthew
• 3D scan of 2.7 meter
statue at 0.25 mm
• 102,868,637 points
• File size: 644 MB
• Preprocessing time:
1 hour
Future Work
• Splats as primitive
– Unify rendering of meshes, volumes, point
clouds
– Compatible with shading after rasterization
– Hybrid point/polygon systems
• High-level visibility / LOD frameworks
– Store different kinds of data at each node:
alpha, BRDF, scattering function, etc.
– Potentially could be used to unify imagebased-rendering (IBR) techniques
Contributions
• Real-time 3D model acquisition system
–
–
–
–
Video-rate 3D scanner for moving objects
Analysis of ICP variants; real-time algorithm
Real-time merging and rendering
Allows user to see model and fill holes
• QSplat: interactive rendering of large 3D
meshes
– Single data structure used for visibility culling,
level-of-detail control, point rendering,
compression
– Extension to network streaming [I3D 2001]
Acknowledgments
• Olaf Hall-Holt
• Lucas Pereira
• The Original DMich Gang: Dave Koller, Sean
Anderson, James Davis, Kari Pulli, Matt Ginzton, Jon
Shade
• DMich, the next generation: Gary King, Steve
Marschner
• Graphics lab
• Advisor: Marc Levoy
• Committee: Pat Hanrahan, Leo Guibas, Mark
Horowitz, Bernd Girod
• Family, friends
• Sponsors: NSF, Interval, Honda, Sony, Intel