Basics of AR VR in italiano
Virtual,
Augmented,
Mixed Reality
Laurea specialistica in Ingegneria Meccanica
Corso di Simulazione e prototipazione virtuale
Ing. Michele Fiorentino
Ing. Giuseppe di Gironimo
m.fiorentino@poliba.it
Ultimo Aggiornamento: 3 febbraio 2009
Politecnico di Bari
Sommario
Il reality-virtuality
Continuum
Interfacce Multimodali
La realtà virtuale
La realtà “aumentata”
La Mixed reality
Reality-Virtuality Continuum,
Milgram
Virtual Reality
Immergere l’utente in un modo
simulato e quindi 3D.
Ci sono diverse tecniche,
compromesso tra:
•livello di immersione
•sensi coinvolti
•Calcolo computazionale
•limiti fisici ed altro.
Virtual Reality: the story
Caratteristiche della VR:
Navigation
Interaction
Immersion
Presence
Virtual Reality
• Navigation is the ability to move around and explore a 3D scene.
• Interaction is the ability to select and move objects in the scene.
• Immersion is achieved by stereoscopic vision.
• Immersion increase the sensation of presence within the virtual
world.
Real-time graphics is fondamental in VR
Real-time graphics
High-speed rendering and interactivity is vital to a VR
system: if there is any delay in the images seen by the
user, the illusion is quickly lost.
Virtual Reality hardware
1. Stereo Viewing
3. Output devices
– Shutter Glasses
– Force
Feedback
– HMD
4. Displays
2. Input devices
– Projection Walls
– Hand Tracking
– Projection Tables
– Gloves Wands
– Full Immersion
– Gestures, Speech
– Autostereoscopic displays
– Holograpy
Virtual Reality: Stereo viewing
VR is that things are presented
in three dimensions.
Humans see using
stereoscopic vision.
In stereoscopic 3D perception,
we use two eyes that are set
apart from each other.
The brain uses this to re-create
depth information.
New York, on
June 10, 1915,
red and green
glasses
Virtual Reality: Stereo viewing
We perceive as objects at
specific locations in the space
around us.
3D graphics display means to
render two images from slightly
different eye points and present
them separately to each eye:
– Shutter Glasses
– Head Mounted Display
Virtual Reality: Stereo viewing
Evoluzione
Infinitec superanaglyph glasses
Philips 42” autostereoscopic
3D display
Virtual Reality: Shutter Glasses
Shutter glasses allow displaying stereo images. The display
alternates rapidly between the left and right eye images.
Each eye only saw the image intended for it by opening a shutter in
front of the eye when its image is being displayed.
The shutters would have to be synchronized
to the display.
Shutter glasses typically use electronic
shutters made with liquid crystals.
Head Mounted Display
Another way to present a separate image to each eye is
to use a separate monitor for each eye
This can be done by mounting small
monitors in some sort of head gear.
Virtual Retina Display
Head Mounted Display
Auto-stereoscopic Displays
Dimension
Technologies, Inc.
Sharp Electronics
Actius RD3D
(The price for the 15" model is
$1,699. The 18" model costs
$6,999. )
www.sharp.com
http://dti3d.com/
Autostereoscopic
Samsung,
Philiphs, etc..
Altri sistemi
Actuality Perspecta® display
works by projecting thousands of
2D images per second onto a
rotating screen.
http://actuality-systems.com/
The Reachin Display
http://www.reachin.se/
Virtual Reality: Input devices
Motion sensors can to track the position of fingers.
Fingers actions can be used to control a program.
This means pushing virtual menu
buttons, or grabbing an object and
moving it around with hands.
Hands and fingers position and
orientation can be achieved by wearing
a glove that has a sensors embedded.
Virtual Reality: Input devices
Immersive tracking systems
Tracking System: ART Dtrack
System Specifications
Optical (IR)
2 Cameras
> 4 markers per body
Volume: 2 x 1.5 x 1.5 m
Cost $$$
Validation
Measured precision: < 1 mm
Virtual Reality: Output devices
•haptic devices return force to the user as feedback
through the tactile senses. Thus it is possible to touch,
weigh and grasp virtual objects.
• The cables and pulleys on
the glove are used to "push
back" at the user under
computer control.
Senstylus: Input\Output devices
Object Collision Feedback
vibrating proximity sensor two different effects
(in/out)
Front-Back phase shift conveys the collision
direction
Projection Walls
Projection Walls
• Group (large) viewing
• medium immersion
Projection Walls
Cylindrical screen
Flat screen
Projection Tables
• Single (Small group ) viewing
• medium immersion
Rooms – Cave Systems
•Single (Small group ) viewing
•Full immersion
Rooms
VR Applications
•
•
•
•
•
•
•
•
VRCAD
Education
Training
Virtual Prototypes
Spatial Visualization
Heritage
Scientific Visualization
Entertainment
VR CAD: Spacedesign
Virtual Reality: Training
VR is used to use expensive equipment or when the
cost of a mistake is very high.
• aircraft simulators to train pilots
• military flight simulators
• medicine (virtual surgery)
• virtual therapy (phobias)
A Boeing 737-800 fullflight
simulator for Lufthansa.
Virtual Reality: Virtual Prototypes
• Reduce physical prototypes
• Improve ergonomics and functionality.
• Provide manufacturing with early access
to product details.
• Collaborative design.
•understanding of complex 3D assemblies
•present product configuration and
aesthetics.
• Crash test
Manufacturing Visualization
VR for exploring 3D environments.
Can solve problems as factory construction or
reorganization:
overall efficiency
machine accessibility
health and safety
services
others …
Virtual Reality: Education
VR enables a sense and motion learning process, more
natural than the symbolic-reconstructive way (ie. writing).
3D graphics is a powerful
tool for visualizing and
understanding complex
and/or abstract information.
Heritage
VR allows:
•access to "monuments" that risk to be
damaged
•reconstruction of "scenarios" that do not
exist anymore
•Time travel
• interaction and avatars
• a vision with no space and time
constrictions.
Scientific Visualization
In Virtual reality participants are immersed in the data rather
than passive observers.
VR vs AR
Arbeiten mit
digitalen Produktdaten
Arbeiten am
realen Produkt
VR
ProduktProduktdefinition
Design
AR
Analyse
ProdukProduktions-tions
planung
ProdukProduktion
Betrieb
Wartung Recycling
Augmented vs. Virtual Reality
Virtual Reality:
Totally immersive
environment
Visual senses are
under control of system
Augmented Reality
System augments the
real world scene
User maintains a sense
of presence in real
world
Needs a mechanism to
combine virtual and
real worlds
Qui
Combining Real and Virtual
Needs:
Precise models
Calibration
Locations and properties of camera and display
Registration
Overlap virtual and real local coordinate systems
Problems
Tracking the objects
Registration
Objects to behave in
physically plausible
manners when
manipulated
Occlusion
Collision detection
Shadows
Performance Issues
Augmented Reality must:
Run in real-time so that
the user can move in
the environment
Show a properly
rendered augmented
image
Display Technologies
Monitor Based
Pico-Projection
Head Mounted Displays:
• Video see-through
• Optical see-through
Monitor Based AR
Simplest available
Little feeling of being
immersed in
environment
Optical see-through HMD
Pros:
Simplicity
Resolution
No eye offset
HMD evolution
Towards true color retinal scanning
Applications
Medical
Entertainment
Military Training
Engineering Design
Robotics and
Telerobotics
Manufacturing,
Maintenance, and
Repair
Consumer Design
Hazard Detection
Audio
Manufacturing & Maintenance
Present to the user:
name of parts
Function
maintenance or manufacturing
records
Etc..
highlighting parts and showing the
path of extraction.
safety information: Parts that are hot
or electrified can be highlighted
Augmented-Reality Application
Medical TAC-MR augmented patient
Laboratorio di Realtà Virtuale
CRCT
Attrezzature:
3 proiettori BARCO 3200x2560 pixel;
Schermo BARCO CADWALL,
6.20x2.20m (il più grande d’europa!!)
occhiali stereoscopici
CRYSTALEYE™
SGI ONYX300 12x CPU- 12 GB RAM
Tracking INTERSENSE IS-900
Sistema di controllo
Software: DIVISION/dVISE (PTC),
MADYMO (TNO), OPEN INVERTOR e
AMIRA (TGS), VIRTUAL DESIGN 2
(VRCOM)
Laboratorio di Realtà Virtuale
CRCT
Fine del Modulo
Domande???
Augmented,
Mixed Reality
Laurea specialistica in Ingegneria Meccanica
Corso di Simulazione e prototipazione virtuale
Ing. Michele Fiorentino
Ing. Giuseppe di Gironimo
m.fiorentino@poliba.it
Ultimo Aggiornamento: 3 febbraio 2009
Politecnico di Bari
Sommario
Il reality-virtuality
Continuum
Interfacce Multimodali
La realtà virtuale
La realtà “aumentata”
La Mixed reality
Reality-Virtuality Continuum,
Milgram
Virtual Reality
Immergere l’utente in un modo
simulato e quindi 3D.
Ci sono diverse tecniche,
compromesso tra:
•livello di immersione
•sensi coinvolti
•Calcolo computazionale
•limiti fisici ed altro.
Virtual Reality: the story
Caratteristiche della VR:
Navigation
Interaction
Immersion
Presence
Virtual Reality
• Navigation is the ability to move around and explore a 3D scene.
• Interaction is the ability to select and move objects in the scene.
• Immersion is achieved by stereoscopic vision.
• Immersion increase the sensation of presence within the virtual
world.
Real-time graphics is fondamental in VR
Real-time graphics
High-speed rendering and interactivity is vital to a VR
system: if there is any delay in the images seen by the
user, the illusion is quickly lost.
Virtual Reality hardware
1. Stereo Viewing
3. Output devices
– Shutter Glasses
– Force
Feedback
– HMD
4. Displays
2. Input devices
– Projection Walls
– Hand Tracking
– Projection Tables
– Gloves Wands
– Full Immersion
– Gestures, Speech
– Autostereoscopic displays
– Holograpy
Virtual Reality: Stereo viewing
VR is that things are presented
in three dimensions.
Humans see using
stereoscopic vision.
In stereoscopic 3D perception,
we use two eyes that are set
apart from each other.
The brain uses this to re-create
depth information.
New York, on
June 10, 1915,
red and green
glasses
Virtual Reality: Stereo viewing
We perceive as objects at
specific locations in the space
around us.
3D graphics display means to
render two images from slightly
different eye points and present
them separately to each eye:
– Shutter Glasses
– Head Mounted Display
Virtual Reality: Stereo viewing
Evoluzione
Infinitec superanaglyph glasses
Philips 42” autostereoscopic
3D display
Virtual Reality: Shutter Glasses
Shutter glasses allow displaying stereo images. The display
alternates rapidly between the left and right eye images.
Each eye only saw the image intended for it by opening a shutter in
front of the eye when its image is being displayed.
The shutters would have to be synchronized
to the display.
Shutter glasses typically use electronic
shutters made with liquid crystals.
Head Mounted Display
Another way to present a separate image to each eye is
to use a separate monitor for each eye
This can be done by mounting small
monitors in some sort of head gear.
Virtual Retina Display
Head Mounted Display
Auto-stereoscopic Displays
Dimension
Technologies, Inc.
Sharp Electronics
Actius RD3D
(The price for the 15" model is
$1,699. The 18" model costs
$6,999. )
www.sharp.com
http://dti3d.com/
Autostereoscopic
Samsung,
Philiphs, etc..
Altri sistemi
Actuality Perspecta® display
works by projecting thousands of
2D images per second onto a
rotating screen.
http://actuality-systems.com/
The Reachin Display
http://www.reachin.se/
Virtual Reality: Input devices
Motion sensors can to track the position of fingers.
Fingers actions can be used to control a program.
This means pushing virtual menu
buttons, or grabbing an object and
moving it around with hands.
Hands and fingers position and
orientation can be achieved by wearing
a glove that has a sensors embedded.
Virtual Reality: Input devices
Immersive tracking systems
Tracking System: ART Dtrack
System Specifications
Optical (IR)
2 Cameras
> 4 markers per body
Volume: 2 x 1.5 x 1.5 m
Cost $$$
Validation
Measured precision: < 1 mm
Virtual Reality: Output devices
•haptic devices return force to the user as feedback
through the tactile senses. Thus it is possible to touch,
weigh and grasp virtual objects.
• The cables and pulleys on
the glove are used to "push
back" at the user under
computer control.
Senstylus: Input\Output devices
Object Collision Feedback
vibrating proximity sensor two different effects
(in/out)
Front-Back phase shift conveys the collision
direction
Projection Walls
Projection Walls
• Group (large) viewing
• medium immersion
Projection Walls
Cylindrical screen
Flat screen
Projection Tables
• Single (Small group ) viewing
• medium immersion
Rooms – Cave Systems
•Single (Small group ) viewing
•Full immersion
Rooms
VR Applications
•
•
•
•
•
•
•
•
VRCAD
Education
Training
Virtual Prototypes
Spatial Visualization
Heritage
Scientific Visualization
Entertainment
VR CAD: Spacedesign
Virtual Reality: Training
VR is used to use expensive equipment or when the
cost of a mistake is very high.
• aircraft simulators to train pilots
• military flight simulators
• medicine (virtual surgery)
• virtual therapy (phobias)
A Boeing 737-800 fullflight
simulator for Lufthansa.
Virtual Reality: Virtual Prototypes
• Reduce physical prototypes
• Improve ergonomics and functionality.
• Provide manufacturing with early access
to product details.
• Collaborative design.
•understanding of complex 3D assemblies
•present product configuration and
aesthetics.
• Crash test
Manufacturing Visualization
VR for exploring 3D environments.
Can solve problems as factory construction or
reorganization:
overall efficiency
machine accessibility
health and safety
services
others …
Virtual Reality: Education
VR enables a sense and motion learning process, more
natural than the symbolic-reconstructive way (ie. writing).
3D graphics is a powerful
tool for visualizing and
understanding complex
and/or abstract information.
Heritage
VR allows:
•access to "monuments" that risk to be
damaged
•reconstruction of "scenarios" that do not
exist anymore
•Time travel
• interaction and avatars
• a vision with no space and time
constrictions.
Scientific Visualization
In Virtual reality participants are immersed in the data rather
than passive observers.
VR vs AR
Arbeiten mit
digitalen Produktdaten
Arbeiten am
realen Produkt
VR
ProduktProduktdefinition
Design
AR
Analyse
ProdukProduktions-tions
planung
ProdukProduktion
Betrieb
Wartung Recycling
Augmented vs. Virtual Reality
Virtual Reality:
Totally immersive
environment
Visual senses are
under control of system
Augmented Reality
System augments the
real world scene
User maintains a sense
of presence in real
world
Needs a mechanism to
combine virtual and
real worlds
Qui
Combining Real and Virtual
Needs:
Precise models
Calibration
Locations and properties of camera and display
Registration
Overlap virtual and real local coordinate systems
Problems
Tracking the objects
Registration
Objects to behave in
physically plausible
manners when
manipulated
Occlusion
Collision detection
Shadows
Performance Issues
Augmented Reality must:
Run in real-time so that
the user can move in
the environment
Show a properly
rendered augmented
image
Display Technologies
Monitor Based
Pico-Projection
Head Mounted Displays:
• Video see-through
• Optical see-through
Monitor Based AR
Simplest available
Little feeling of being
immersed in
environment
Optical see-through HMD
Pros:
Simplicity
Resolution
No eye offset
HMD evolution
Towards true color retinal scanning
Applications
Medical
Entertainment
Military Training
Engineering Design
Robotics and
Telerobotics
Manufacturing,
Maintenance, and
Repair
Consumer Design
Hazard Detection
Audio
Manufacturing & Maintenance
Present to the user:
name of parts
Function
maintenance or manufacturing
records
Etc..
highlighting parts and showing the
path of extraction.
safety information: Parts that are hot
or electrified can be highlighted
Augmented-Reality Application
Medical TAC-MR augmented patient
Laboratorio di Realtà Virtuale
CRCT
Attrezzature:
3 proiettori BARCO 3200x2560 pixel;
Schermo BARCO CADWALL,
6.20x2.20m (il più grande d’europa!!)
occhiali stereoscopici
CRYSTALEYE™
SGI ONYX300 12x CPU- 12 GB RAM
Tracking INTERSENSE IS-900
Sistema di controllo
Software: DIVISION/dVISE (PTC),
MADYMO (TNO), OPEN INVERTOR e
AMIRA (TGS), VIRTUAL DESIGN 2
(VRCOM)
Laboratorio di Realtà Virtuale
CRCT
Fine del Modulo
Domande???