Visual Behavior Based Bio-inspired Polarization Techniques in Computer Vision and Robotics

  

Dr. Mohammad Iqbal

Visual Behavior Based Bio-inspired Polarization

Techniques in Computer Vision and Robotics

  Disajikan dalam seminar : “Further image processing Applications: in the natural sciences

and medicine”

  Rabu, 5 Desember 2012

Sistem Pencitraan Non konvensional terinspirasi dari Alam

Sistem Pencitraan Non konvensional terinspirasi dari Alam Agenda y

  Introduction Polarization y

  Bio-inspired polarization vision : y

  Pola-Stereo Vision y

  Omnidirectional Vision and more Applications y

  Camera calibration (Polarization + caustics) and shape from polarization y

  Robot navigation (Skylight polarization patterns) y

  UAV attitude estimation (Polarimetric segmentation) y

  Multiple Spectral, UV and IR

  Introduction Intensity

The  fact of light in Nature

  Wave length – color Polarization

  Polarization y

  Polarization is the phenomenon that describes the oscillations orientation of the light (or other radiation) waves which are restricted in direction. y

  Humans need the help of polarizing optics to visualize most invisible polarization effects.

  Polarization

y Light can be polarized by several processes :

• Selective Absorption – Dichroism • Reflection • Scattering • Birefringent

Bio-inspired polarization vision y

  Many fish, cephalopods, crustaceans, insects, and other animals are capable of perceiving polarized light. y

  Polarization vision can be used for most tasks of color vision like: object

recognition, contrast enhancement, camouflage breaking, and signal

detection and discrimination.

  From left to right (aquatic insects): a) Dragonfly b) Bio-inspired polarization vision y Useful for navigation tasks : y Self Localization y Orientation y Detection of water, food, prey or obstacle y Bio-inspired examples : y terrestrial animal y visual ability to analyze light pattern in the sky or in the . reflected surfaces y marine animals y

  Camouflage or communication y

  Enhance the visibility of the scenes Bio-inspired polarization vision y Terresterial Animal

  Bee Visual Ability : Self Localization and Orientation info to find a food

  In addition to their ability to sense Earth's magnetic

  field, the bees also have Polarization sensitive ability. This can be understood from the fact that bees

  have a unique behaviour based-on the position of the sun and the earth's gravity called the swaggle waggle

  dance of bees [Srinivasan (2011)]

  Based-on review of [Andreou and Kalayjian (2002)], other terrestrial animals such as dragonflies and waterstrides use the polarization of reflected light to detect water. The insec alike beetles, flies, and also some reptile such as salamanders and lizards are able to sense and use polarization in their environments.

  Bio-inspired polarization vision The Octopus used polarization y Marine Animal

  information to increase visual contrast

  The cephalopods (one kind of squid) have

  [Shashar and Cronin (1996)]. they have more unique capability to sense polarization of a unique colour-vision system that is light. They can respond and reflect the patterns of able to register a change, located in the polarized light at once. The interesting fact is, the mid-band region of the eye. The mid- cephalopods are colorblind, but their eyes have band may therefore be involved in

  photoreceptors and corresponding hair-like

  communication in both colour and

  microvilli which expand their surface area, and

  polarization space [Marshall and can enhanced ability to selectively perceive Shashar (1999)]. linearly polarized light [Shashar and Cronin (1996)].

  The mantis shrimp have a hyperspectral eyes with

  Cuttlefish (one species of cephalopod)

  crystalline structure of their have ability to manipulate the reflected pattern of

  microvilli which perceive

  polarization in their skin. They can use its ability from the infra-red, visible to for camouflage and also for enhancing the

  Bio-inspired Stereo vision

Human eyes are horizontally separated by about 50–75 mm (interpupillary distance) depending on each individual

  y Interpupillary distance (IPD) is the distance between the center of the pupils of the two eyes. y

  Based on Anthropometric databases in y Military Handbook 743A y

  J. O. Merritt, S. A. Benton and M. T. Bolas (eds.), Proceedings of SPIE: Stereoscopic Displays and Virtual Reality Systems XI, Vol. 5291, pp. 36-46. San Jose CA) y

  Smith, G., & Atchison, D. A. (1997). The eye and visual optical instruments. Cambridge UK: Cambridge University Press.

  Most of animal (also human) have a stereo capabilities to aware scene around.

  Human eyes have an overlapping field of view of about 120º. It is only Pola-Stereo Vision

The Challenge

  y Polarization Imaging : The images are ‘darker’ than intensity images, need at least three different images. y

Stereo vision : matching point

  problem, need clear images, different view, need more geometric approach.

  HOW TO GET A WIN-WIN COMBINATION? Pola-Stereo Vision y

To build a prototype of stereo vision The Objective system with polarization sensitivity :

  y Can measure DoP and AoP for every angle of incident light. y

  Can reconstruct 3D point of stereo images y

To develop a simple and fast polarization imaging algorithm based- on stereo vision

  y Simple and Efficient in setup and algorithm y

  Easy to Use Pola-Stereo Vision Learning from Previous Researches

  BeamSplitter

  y Increase the polarization parameters estimation

  y No 3D information

  Wolff et al (1990, 1994, 1995)

  y Stereo video polarimetry system to visualize the

  2

  polarization patterns in

  Mizera et al 2001

  stereovision

  1 camera

  y Displacement of the camera y Two images are taken simultaneously with different

  polarization filter settings Pola-Stereo Vision Principle of Design y Stereovision => 2 cameras y Measurement of partially linearly polarized light => at least need 3 images y Automatic acquisition => Liquid crystal components

  Pola-Stereo Vision

Polarization-Stereo Imaging System Schema

  Image Polarization Acquisition

  Calibration Calibration Rectifying Image

  Stereo Geometric Calibration

Stereo

  Feature Detection Evaluation

Calibration

  Stereo Matching Remove

  3D

  3D Outlier Reconstruction

  Reconstruction Pola-Stereo Vision Evaluation of System Capabilities

In nature, the light reflected from real objects would have

  Heterogeneous Scene many variations in orientations.

  3D Reconstruction Omnidirectional Vision Introduction _{Blue Sky Robot with the sensitively polarization camera} y

  Blue sky contains a skylight polarization pattern. y

  Some insects (e.g., Bee, Cricket, and Desert ant Cataglyphis) have ability to utilize the polarization

  To estimate polarization pattern of pattern of the sky for navigation skylight purpose. y

  Estimating Polarization pattern a polarization

  The purpose is to use pattern can be applied for skylight polarization pattern for navigation navigation robot by following the Omnidirectional Vision Learning from Previous Researches

• Some insects have ability to

  y Implementing a polarization pattern as utilize polarization pattern for robot. [Lambrinos additional way in navigation navigation.

  et al.,2000]

  1. Desert ant Cataglyphis is able to explore its desert habitat y

  The polarization pattern has the same for hundreds of meters while pattern between simulation model and the foraging and return back to its real experiment results, it has been shown nest precisely on a straight line

  by taking skylight polarization pattern . [ Kesson et al.,2002]

  [Pomozi et al.,2001] : by Pomozi.

  2. Cricket is able to use skylight polarization under low degree of polarization for orientation even under unfavorable meterological

  The insect’s conditions. [ Labhart, 1996] selected region Omnidirectional Vision Sensor and Optical System y

  The system uses Polarimetric camera and mirror of Various Type of catadioptric.

  Wide area Lens System y

  Catadioptric has an omnidirectional view Camera Polarizer Mirror

  Omnidirectional Vision Pola-Catadioptric Schema

  Determine Sun location Determine Sun location

  Selecting a mirror for a camera sensor Selecting a mirror for a camera sensor

  Hyperbolic Hyperbolic

  Parabolic Parabolic

  Spherical Spherical

  Rotating a camera sensor by using Roll and Yaw rotations system

  Rotating a camera sensor by using Roll and Yaw rotations system

  X-axis rotation (Roll) X-axis rotation (Roll)

  Z-axis rotation (Yaw) Z-axis rotation (Yaw)

  Combination Yaw + Roll rotations Combination Yaw + Roll rotations

  Estimation of Estimation of Polarization The combination of rotation system Omnidirectional Vision

Evaluation of System Capabilities

  Yaw  = 0 Yaw  =  +45 Yaw  =  ‐45 Roll  = +30 Roll  = ‐30 Omnidirectional Vision More Application… y

  Shape from Polarization y

  Camera calibration

  3D Reconstruction by means of polarization imaging y Robot navigation

  Simulation of the sky angle of polarization pattern in Le Hegedüs et al., Polarization patterns of thick clouds: overcast skies have distribution of the angle of polarization similar to that of clear skies, J.Opt.Soc.Am.A, 2007.

  

Omnidirectional Vision

More Application…

  Omnidirectional Vision More Application… y

  UAV ( unmanned aerial vehicle ) attitude estimation y Multiple Spectral

  Multiple Spectral, UV and IR

Multi-spectral (Multi = more than one, Spectral = plural of spectrum)

  Spectrum = A band of colors, as seen in a rainbow, produced by separation of the components of light using prism by their different degrees of refraction according to wavelength.

  Asclepios system Shape from Shading Shape from Polarization φ

  

Multiple Spectral, UV and IR

y

  Multiple Spectral Shape from Shading Shape from Polarization

  

Multiple Spectral, UV and IR

y

  Shape from UV UV ( ultraViolet ) – Shape from UV y

  IR ( Infra Red ) – Scanning From Heating Scanning From Heating

  

Multiple Spectral, UV and IR

  Measurement and Calibration y

  Stokes Polarimeter Design Calibration Step :

  Two measurement of pol states of the scenes : He-Ne Laser beam with output a single • Stokes Vector : wavelength at 632.8 nm (Main Focus), S0 : Total intensity ( I0 + I90 )

• An observed real scene made of a monitor display S1 : Difference between horizontal and vertical linear and a single polarizer oriented at 0°.

  polarization ( I0 - I90 ) S2 : Difference between linear polarization at 45° Measurement Result : Conclusion

Bio Inspired of Polarization Light Research can be classified into :

  y Based-on Combine the two systems : 1.

  Stereo + Polarization vision (iqbal, 2010) 2. Omnidirection (Wide area Lens system) + Polarization vision (Puja, 2010) y

  Based-on result : 1.

  3D recontruction (iqbal, 2010) 2. Navigation 3. Shape or scene estimation 4. Measurement and Calibration (Aristio, 2011) y

  Based-on light Spectrum : 1.

  Visible light Publication y

  

M. Iqbal, O. Morel, and F. Meriaudeau. A survey on outdoor water hazards detection. In the 5th International Conference on Information &

Communication Technology and Systems (ICTS), 2009. y

  

M. Iqbal, O. Morel, and F. Meriaudeau. Choosing local matching score method for stereo matching based-on polarization imaging. In

Computer and Automation Engineering (ICCAE), 2010 The 2nd International Conference on, volume 2, pages 334 –338, feb. 2010. doi: 10.1109/ICCAE.2010.5451536. y

  M. Iqbal, O. Morel, and F. Meriaudeau. Extract information of polarization imaging from local matching stereo. In Intelligent and Advanced Systems (ICIAS), 2010 International Conference on, pages 1 –6, june 2010. y

  M. Iqbal, F. Mériaudeau, and O. Morel. Polarization stereoscopic imaging prototype. International Journal of Signal Processing, Image Processing and Pattern Recognition (IJSIP) SERSC, 4 (3), September 2011. y Abd El Rahman Shabayek, Olivier Morel, and David Fofi. Polarization in the eye of a catadioptric sensor. Unpublished work, 2011. y

  

Istvan Pomozi, Gbor Horvth, and Rdiger Wehner. How the clear-sky angle of polarization pattern continues underneath clouds: full-sky

measurements and implications for animal orientation. Experimental Biology, 204:29 3 3 –2942, June 2001. y

  

Lianghai Wu, Jun Gao, Zhiguo Fan, and Zhao Xie. How to get navigation information within patches of sky as insect do? International

Conference on Signal Processing Systems (ICSPS), (2), 2010. y

  T. Labhart. How polarization-sensitive interneurones of crickets perform at low degrees of polarization. Experimantal Biology, 199:1467– 1475, March 1996. y

  Sussane Kesson and Rdiger Wehner. Visual navigation in desert ants cataglyphis fortis: are snapshots coupled to a celestial system of reference? Experiment Biology, pages 1971–1978, 2002. y

  

Kane Usher, Peter Ridley, and Peter Corke. A camera as a polarized light compass: Premliminary experiments*. pages 116–120, Sydney,

November 2001. Australian Conference on Robotics and Automation. y

  Abd El Rahman SHABAYEK, Cédric DEMONCEAUX, Olivier MOREL, David FOFI, "Vision Based UAV Attitude Estimation: Progress and Insights", Accepted for publication in the International Conference on Unmanned Aircraft Systems (ICUAS'11), Denver, CO USA, May 2011.

  Thanks ☺