Automatic road sign identification system with robustness to partial occlusion.
AUTOMATIC ROAD SIGN IDENTIFICATION SYSTEM
WITH ROBUSTNESS TO PARTIAL OCCLUSION
BY
NURSABILLILAH BINTI MOHD ALI
A dissertation submitted in fulfilment of the requirement for
the degree of Master of Science in
Mechatronics Engineering
Kulliyyah of Engineering
International Islamic University Malaysia
JANUARY 2014
© Universi · Teknikal Malaysia Melaka
ABSTRACT
In recent years, automatic road sign identification system has attracted numerous
research works with the possibility of using in autonomous or driver assistance system
(ADAS). Research in road sign identification with occlusion, however is still lacking.
Many existing techniques up to now that have been developed algorithms with the
existence of occlusions produce inaccuracy that needs to be improved. Even though
the occurrences of road signs with presence of occlusion are small, yet it is problem
that needs to be addressed. An intelligent system for road sign identification that
incorporated several different algorithms is proposed in this research to solve the
problems. The algorithms consist of proposed HSV and RGB colors in detection part
and ANN and PCA techniques in recognition part. The proposed algorithms are able
to detect the three colored images of road sign namely Red, Yellow and Blue. These
algorithms are then compared with each other to evaluate their performance. The
hypothesis of this research is that road sign images can be used to detect and identify
signs involved existence of occlusions and rotational changes. Each sign features are
extracted using global feature extraction technique whereby the vertical and
dimension size of sign are fixed to a standard size. These input features are used to be
applied into neural network according to feed forward neural network technique using
backpropagation training function. The sign image can be easily identified by the PCA
method as it has been used in many application areas. Based on the experimental
result, it shows that the HSV is robust in road sign detection with minimum of 88%
and 77% successful rate for non-partial and partial occlusions images rather. For
successful recognition rates using ANN can be achieved starts from 75-92% whereas
PCA is in the range of 94-98%. The combination of HSV color-based detection and
PCA generated faster processing time of 2.1 s per frame for the overall identification
process. The occurrences of all classes are recognized successfully is between 5% and
10% level of occlusions using PCA, whereas only 5% level of occlusions successful
recognized using ANN.
11
ｾｌｬ＠
t: ｯｾ＠
Ｌｾ｜ＮｌＭｪ＠
ｾ＠
Jwi J):JI ＴＭｾ＠
t: J:):JI ＭＺｶｾ＠
ｾｙＧｊ＠
ｾ＠
ｾ＠
C:: uyj)y=- ｾ｟Ｌ＠
ｾ｟Ｌ＠
ｾ｟［Ｎｩ｜Ｑ＠
ｾｊＮ＾､＠
ｾ＠
ｬＮｲｾ＠
｟ｲｾｬ＠
ｾ＠
ｾｬ＠
ｾ｜＠
J)\;.c if ｾ＠
＼ｬＮｩｾ＠
ｊｾ＠
ｾ＠
Ｌｾ｜＠
ｾｗｉ＠
J.r::)l ｾﾷｩｦＡＮ＠
, ｐｃａｩｬｾＺ＠
ｾｴＮ［＠
ｾｷ＠
ｾ＠
J..w_,
'·-=··-=u..t:li ｾ＼ＮＱ＠
｢ｾｬＮ＠
!.
ｊｾ＿Ｇ［ｬ＠
ｾ＾Ｍ
ａｎＮｩｬｾＺ＠
｣Ｎｳｾ＠
ｾ＠
ｾ＠
if
ｾ＠
ｊｾ＠
ｾｴＮ［＠
111
ｾ＠
ｾ＠
if Ｇ［Ｑｾ＠
ｾ＠
ｾＭｬｩ＠
il....k)'
ｩｾ＠
ｾ＠
{
w- PCA ｾ［Ｎ＠
ｾ｟Ｌ［＠
HSV
.Ji
J;r
ｾ｟Ｌ＠
ｾ＠
w_,
J.>-
ｾ｜ＮｌＭｪ＠
0iANN
cr. if Q\..l>- ｾ＠ J.>ｵｾｬ＠
ｾＱＭＮｬｪ｜＠
｣ｾ＠
if ｾ＠
ｾｲＭ
75
｣ｾ＠
J_,._.dl
Jl Ｇ［ｶｾ＠
ｾ＠
ａＮ＼ｬｾ｜＠
ＹＲｾ＠
ｾ＠
y
ｯ｟Ｉｾｵ＠
fo Jl o_r..\.1 ｣ｾｬ＠
ｾ＠
PCA .:._?- ｾＡＮ＠
!. 10 _, !. ＵＮＺ｟ｾ＠
J:):JI ＭＺｶｾ＠
ｾ＠
ｾ＠
RGB
ＰＱｽｾ＠
o.l.a. ｾｊ＠
ｵｾｬ＠
JJ-1 t: J:):JI ＭＺｶｾ＠
.J,JJI ｾｦｩ＠
ｾ｟Ｉｩ＠
C:: ｵｹｪＩｾｬ＠
ｾ＠
ｾＮｗｉ＠
ｾ｜ＮｌＭｪ＠
J.>-
ｩｬｾＺ＠
ｾ＠
ｾ｜ＮｌＭｪ＠
RGB
ｾＨｬ＠
ｾ＠
ｾ＠
c_I/:JI .:./' ｾ｜＠
ｾｹ＠
ＭＺＮ［Ｇｽｾｩｦ＠
88if ＭＮＺｰｾｬ＠
>-7- ｾ＠
JJ_,
';}
uyj)y=- ｯｾ＠
W::;;!
ｊｰｾ＠
ＭＺｶｾ＠
ｾﾷ＠
ｾ｜＠
ｾｬ｢＠
ｾ＠
ｩｬｾＺ＠
¥_,
ｾ＠
ｾｉＮｌＭｪｬ＠
l::-' Ｌｯｾ＠
o.l.a. ｩｾｊﾷ＠
-:.;';J..hl ｬＮｲｾﾷ＠
ＹＴＭＸｾ｟Ｌｬ＾
ｾ＠
ｾ｜＠
ｾ＠
.;;)\;.c...U\ Ｎ［ｾ｜＠
Jl
ｩｾ＠
ｾ＠
J.>-
ｊｾ＠
Jl (l): Jl ｾＮｬ＠
ｾ｜＠
Ｎ［ｾ＠
i\.6 ｩ｜ｾＱ＠
ｾ＠
Ｎ［ｾ｟Ｌ＠
ｾ＠
{ Jl oJ_,._.dl ｾ＠
!. 77 ｣ｾ＠
ｩｬｾＺ＠
ｾｩ＠
o_r..\.1 ｣ｾｬ＠
ｾ＠
w-.
J.>-
-:.;1_,.:-JI ｾ＠
ｯｾｗｉ＠
ｾｹ＠
ｾ＠
Jl (l):
_)_,._.dl if ｾ｜＠
ｾ＠
ｾ＠
ｾ＠
PCA J_,r-o _r
ｊｾ＠
ｾ｟Ｌ＠
ｾ＠
4-y\ ｾ＠
ｾ｟ＬｊＮｉ＠
\..La. ｾＩＮ＠
ｾ｜ＮｌＭｪ＠
ｾＮｲＺＭｩ＠
_, ANN ｾ＼ｊｉ＠
ｾＭＧ＠
d y ｾ｜＠
(. y_,\;::.\1 Ｎ［｜｟ｲＺ､ｾ＠
ｾｩＩＱ＠
ｾｲＭｐｃａ＠
ｾ＠
ｩ｜ｾＱ＠
;;J.,.L...
J5l.:J.I ｾ＠
ｊｾ＿Ｇ［ｬ＠
ｊ｟Ｉｪｾ＠
(.
if ｾＮｗｉ＠
J.>-.
i\.6 ｾｊﾷ＠
0.r- ｾ＠
ifbi (..;JL.. _,i JI..U\ r--0-1 ｾ＠
ｑｾｩｹｬ＠
HSV if
ｾ｟［Ｎｩ｜Ｑ＠
ｾＱＩＮ＠
ｾ＠
Jl .J'Jil ._s>- ｾＸＮ｜＠
C:: J:):JI Ｎ［ｾｩｦ＠
Jl J:):JI ＭＺｶｾ＠
ＭＺｶｾ＠
if ｣ＮｳｾｬＡ＠
Ｕｾ＠
ｾ＠
ｾ＠
APPROVAL PAGE
I certify that l have supervised and read this study and that in my opinion; it conforms
to acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Master of Science (Mechatronics
c---
Engineering).
I certify that I have read this study and that in my opini it conforms to acceptable
standards of scholarly presentation and is fully adequate, in scope and quality, as a
dissertation for the degree of Master of Science (Mechatronics Engineering).
....［ｬｾｩﾷｳ＠ !.Y.Y.1v....
Arnir
Internal Examiner
Hadzli Hashim
External Examiner
This dissertation was submitted to the Department ofMechatronics Engineering and is
accepted as a fulfilment of the requirement for ｴｾ･＠
d ee of ｍｾｳｴ＠
r of Science
)_,,\J
(Mechatronics Engineering).
,
gr.
.
-
l
l-
········· ﾷＺｾｺ［Ｍ＠
ｾＯ＠
--
WITH ROBUSTNESS TO PARTIAL OCCLUSION
BY
NURSABILLILAH BINTI MOHD ALI
A dissertation submitted in fulfilment of the requirement for
the degree of Master of Science in
Mechatronics Engineering
Kulliyyah of Engineering
International Islamic University Malaysia
JANUARY 2014
© Universi · Teknikal Malaysia Melaka
ABSTRACT
In recent years, automatic road sign identification system has attracted numerous
research works with the possibility of using in autonomous or driver assistance system
(ADAS). Research in road sign identification with occlusion, however is still lacking.
Many existing techniques up to now that have been developed algorithms with the
existence of occlusions produce inaccuracy that needs to be improved. Even though
the occurrences of road signs with presence of occlusion are small, yet it is problem
that needs to be addressed. An intelligent system for road sign identification that
incorporated several different algorithms is proposed in this research to solve the
problems. The algorithms consist of proposed HSV and RGB colors in detection part
and ANN and PCA techniques in recognition part. The proposed algorithms are able
to detect the three colored images of road sign namely Red, Yellow and Blue. These
algorithms are then compared with each other to evaluate their performance. The
hypothesis of this research is that road sign images can be used to detect and identify
signs involved existence of occlusions and rotational changes. Each sign features are
extracted using global feature extraction technique whereby the vertical and
dimension size of sign are fixed to a standard size. These input features are used to be
applied into neural network according to feed forward neural network technique using
backpropagation training function. The sign image can be easily identified by the PCA
method as it has been used in many application areas. Based on the experimental
result, it shows that the HSV is robust in road sign detection with minimum of 88%
and 77% successful rate for non-partial and partial occlusions images rather. For
successful recognition rates using ANN can be achieved starts from 75-92% whereas
PCA is in the range of 94-98%. The combination of HSV color-based detection and
PCA generated faster processing time of 2.1 s per frame for the overall identification
process. The occurrences of all classes are recognized successfully is between 5% and
10% level of occlusions using PCA, whereas only 5% level of occlusions successful
recognized using ANN.
11
ｾｌｬ＠
t: ｯｾ＠
Ｌｾ｜ＮｌＭｪ＠
ｾ＠
Jwi J):JI ＴＭｾ＠
t: J:):JI ＭＺｶｾ＠
ｾｙＧｊ＠
ｾ＠
ｾ＠
C:: uyj)y=- ｾ｟Ｌ＠
ｾ｟Ｌ＠
ｾ｟［Ｎｩ｜Ｑ＠
ｾｊＮ＾､＠
ｾ＠
ｬＮｲｾ＠
｟ｲｾｬ＠
ｾ＠
ｾｬ＠
ｾ｜＠
J)\;.c if ｾ＠
＼ｬＮｩｾ＠
ｊｾ＠
ｾ＠
Ｌｾ｜＠
ｾｗｉ＠
J.r::)l ｾﾷｩｦＡＮ＠
, ｐｃａｩｬｾＺ＠
ｾｴＮ［＠
ｾｷ＠
ｾ＠
J..w_,
'·-=··-=u..t:li ｾ＼ＮＱ＠
｢ｾｬＮ＠
!.
ｊｾ＿Ｇ［ｬ＠
ｾ＾Ｍ
ａｎＮｩｬｾＺ＠
｣Ｎｳｾ＠
ｾ＠
ｾ＠
if
ｾ＠
ｊｾ＠
ｾｴＮ［＠
111
ｾ＠
ｾ＠
if Ｇ［Ｑｾ＠
ｾ＠
ｾＭｬｩ＠
il....k)'
ｩｾ＠
ｾ＠
{
w- PCA ｾ［Ｎ＠
ｾ｟Ｌ［＠
HSV
.Ji
J;r
ｾ｟Ｌ＠
ｾ＠
w_,
J.>-
ｾ｜ＮｌＭｪ＠
0iANN
cr. if Q\..l>- ｾ＠ J.>ｵｾｬ＠
ｾＱＭＮｬｪ｜＠
｣ｾ＠
if ｾ＠
ｾｲＭ
75
｣ｾ＠
J_,._.dl
Jl Ｇ［ｶｾ＠
ｾ＠
ａＮ＼ｬｾ｜＠
ＹＲｾ＠
ｾ＠
y
ｯ｟Ｉｾｵ＠
fo Jl o_r..\.1 ｣ｾｬ＠
ｾ＠
PCA .:._?- ｾＡＮ＠
!. 10 _, !. ＵＮＺ｟ｾ＠
J:):JI ＭＺｶｾ＠
ｾ＠
ｾ＠
RGB
ＰＱｽｾ＠
o.l.a. ｾｊ＠
ｵｾｬ＠
JJ-1 t: J:):JI ＭＺｶｾ＠
.J,JJI ｾｦｩ＠
ｾ｟Ｉｩ＠
C:: ｵｹｪＩｾｬ＠
ｾ＠
ｾＮｗｉ＠
ｾ｜ＮｌＭｪ＠
J.>-
ｩｬｾＺ＠
ｾ＠
ｾ｜ＮｌＭｪ＠
RGB
ｾＨｬ＠
ｾ＠
ｾ＠
c_I/:JI .:./' ｾ｜＠
ｾｹ＠
ＭＺＮ［Ｇｽｾｩｦ＠
88if ＭＮＺｰｾｬ＠
>-7- ｾ＠
JJ_,
';}
uyj)y=- ｯｾ＠
W::;;!
ｊｰｾ＠
ＭＺｶｾ＠
ｾﾷ＠
ｾ｜＠
ｾｬ｢＠
ｾ＠
ｩｬｾＺ＠
¥_,
ｾ＠
ｾｉＮｌＭｪｬ＠
l::-' Ｌｯｾ＠
o.l.a. ｩｾｊﾷ＠
-:.;';J..hl ｬＮｲｾﾷ＠
ＹＴＭＸｾ｟Ｌｬ＾
ｾ＠
ｾ｜＠
ｾ＠
.;;)\;.c...U\ Ｎ［ｾ｜＠
Jl
ｩｾ＠
ｾ＠
J.>-
ｊｾ＠
Jl (l): Jl ｾＮｬ＠
ｾ｜＠
Ｎ［ｾ＠
i\.6 ｩ｜ｾＱ＠
ｾ＠
Ｎ［ｾ｟Ｌ＠
ｾ＠
{ Jl oJ_,._.dl ｾ＠
!. 77 ｣ｾ＠
ｩｬｾＺ＠
ｾｩ＠
o_r..\.1 ｣ｾｬ＠
ｾ＠
w-.
J.>-
-:.;1_,.:-JI ｾ＠
ｯｾｗｉ＠
ｾｹ＠
ｾ＠
Jl (l):
_)_,._.dl if ｾ｜＠
ｾ＠
ｾ＠
ｾ＠
PCA J_,r-o _r
ｊｾ＠
ｾ｟Ｌ＠
ｾ＠
4-y\ ｾ＠
ｾ｟ＬｊＮｉ＠
\..La. ｾＩＮ＠
ｾ｜ＮｌＭｪ＠
ｾＮｲＺＭｩ＠
_, ANN ｾ＼ｊｉ＠
ｾＭＧ＠
d y ｾ｜＠
(. y_,\;::.\1 Ｎ［｜｟ｲＺ､ｾ＠
ｾｩＩＱ＠
ｾｲＭｐｃａ＠
ｾ＠
ｩ｜ｾＱ＠
;;J.,.L...
J5l.:J.I ｾ＠
ｊｾ＿Ｇ［ｬ＠
ｊ｟Ｉｪｾ＠
(.
if ｾＮｗｉ＠
J.>-.
i\.6 ｾｊﾷ＠
0.r- ｾ＠
ifbi (..;JL.. _,i JI..U\ r--0-1 ｾ＠
ｑｾｩｹｬ＠
HSV if
ｾ｟［Ｎｩ｜Ｑ＠
ｾＱＩＮ＠
ｾ＠
Jl .J'Jil ._s>- ｾＸＮ｜＠
C:: J:):JI Ｎ［ｾｩｦ＠
Jl J:):JI ＭＺｶｾ＠
ＭＺｶｾ＠
if ｣ＮｳｾｬＡ＠
Ｕｾ＠
ｾ＠
ｾ＠
APPROVAL PAGE
I certify that l have supervised and read this study and that in my opinion; it conforms
to acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Master of Science (Mechatronics
c---
Engineering).
I certify that I have read this study and that in my opini it conforms to acceptable
standards of scholarly presentation and is fully adequate, in scope and quality, as a
dissertation for the degree of Master of Science (Mechatronics Engineering).
....［ｬｾｩﾷｳ＠ !.Y.Y.1v....
Arnir
Internal Examiner
Hadzli Hashim
External Examiner
This dissertation was submitted to the Department ofMechatronics Engineering and is
accepted as a fulfilment of the requirement for ｴｾ･＠
d ee of ｍｾｳｴ＠
r of Science
)_,,\J
(Mechatronics Engineering).
,
gr.
.
-
l
l-
········· ﾷＺｾｺ［Ｍ＠
ｾＯ＠
--