Detection of partially occluded human using separate body parts classifiers.
DETECTION OF PARTIALLY OCCLUDED HUMAN
USING SEPARATE BODY PARTS
CLASSIFIERS
BY
NURUL FATIHA BINTI JOHAN
INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
2015
© Universiti Teknikal Malaysia Melaka
DETECTION OF PARTIALLY OCCLUDED HUMAN
USING SEPARATE BODY PARTS
CLASSIFIERS
BY
NURUL FATIHA BINTI JOHAN
A dissertation submitted in fulfilment of the requirement
for the degree of Master of Science in Mechatronics
Engineering
Kulliyyah of Engineering
International Islamic University Malaysia
MAY 2015
© Universiti Teknikal Malaysia Melaka
ABSTRACT
The application of computer vision in the surveillance system has provided huge
advantages in the field of security and safety system. In recent years, human detection
and classification subjects have shown an increasing focus in finding specific
individual such as in the case of detecting person in crowded places at a time.
Detection and classification of human can be a challenging task due to the wide
variability of human appearance in terms of clothing, lighting conditions and the
occlusion. These constraints directly influence the effectiveness of the overall system.
To cope with these problems, human detection and classification system is presented
in this thesis which requires fast computations in addition of accurate results. The
propose system will first detect the human in an image by using YCbCr color
thresholding for skin color detection algorithm and then classify the body parts using
artificial intelligent neural network classifier into specific class and finally extend the
classification system with the majority voting technique in order to improve the
classification performance.The first hypothesis of the research is that YCbCr skin
color detection method can be used to detect and identify the exposed human body
parts even with the existence of various illumination conditions and complex
background. In this work, the body parts then only cover face and hands. The body
features are then extracted using feature extraction technique with the dimension of
region detected fixed to a standard size.These body features are then used as an input
to neural network system in order to classify the body parts into specific class.
Meanwhile each class consists of three classifier which is taken from the extracted
body regions and separated into face classifier, right hand classifier and left hand
classifier. Finally, the results of each body parts classification will be processed using
majority voting technique for overall conclusion of the classification system which is
robust to partial occlusion. Experimental results indicate that the human detection
using YCbCr color space is capable to detect the human body with the percentage of
face detection is 92%, right hand detection is 86% and left hand detection is 85%.
Meanwhile the performance of ANN classification system is successful in identifying
face, right hand and left hand which are 90%, 73% and 74% respectively. Whereas,
the accuracy of all 9 classes (Class A until Class I) is found to be 43% and highest to
be 95%. Based on the extended classification system using majority voting technique,
the results have shown a bit improvement on the classification performance for all 9
classes which is the lowest is increase to 45% and the highest is increase to 100%.
ii
خاصة البحث
إن التطبيقات عبالكمبيوتر ي نظام امراقبة وفرت مزاا هائلة ي جال أنظمة اأمن والسامة .ي الس وات
اأخرة ،أظهرت الكشف البشري واموضوعات اأخري أظهرت زادة ي القدرة علي تص يف الركيز ي
العثور على شخص معن ح لو كان ي اأماكن امزدمة ي الوقت نفسه .إن الكشف والتص يف البشري
مكن أن يكون مهمة صعبة نظرا للتقلبات الواسعة ي مظهر اإنسان من حيث امابس و ظروف اإضاءة و
انسداد الرؤا .هذ القيود تؤثر أثرا مباشرا على فعالية ال ظام العام.وابد من استخدام الكشف وتص يف
ال ظام البشري ي هذ اأطروحة ال تتطلب حساات سريعة اإضافة إ ال تائج الدقيقة .سيقوم ال ظام
الكشف أوا عن اإنسان ي شكل صورة استخدامعاةة اللون واإدرا اسسيمع خوارزمية الكشف ومن ثم
تصمف أجزاء اةسم استخدام الذكاء ااصط اعي وتص فهم لفئة معي ة ،وأخرا توسيع نظام التص يف مع تق ية
التصويت من أجل حسن اأداء ي التص يف وجعل امص ف أقوي من احية اانسداد اةزئي.ال ظام امقرح
يستخدم وفقخوارزمية الكشف عن لون البشرةوالشبكة العصبية ونظام الذكاء ااصط اعي حيث ب يت على
تسعةمواضع ي اةسمالبشري .والفرضية اأو من ي البحث اسا هي أن معاةة اللون واإدرا اسسي
طريقة للكشف عن لون البشرة مكن استخدامها للكشف و حديد أجزاء من جسم اإنسان ح مع وجود
الظروف امختلفة كاإضاءة واخلفية الغر واضحة .ي هذا البحث ،امهم من أجزاء جسم اإنسان الوجه
واليدين فقطومن ث يتم توضيخ اميزات استخدام تق ية التميز و البعد من م طقة اكتشاف ابتة إ حجم
يتم استخدام ميزات اةسم كمدخل ل ظام الشبكة العصبية من أجل تص يفججزاء اةسم إ فئات
قياسي .ث م
معي ة .إضافة ا ذلك ،يتكون كل ص ف من ثاثة أص اف اخذت من م اطق ي اةسم وص فت وفق اآي،
ستتم معاةة نتائج كل التصانيف أجزاء اةسم استخدام
اليد اليمى و اليسرى من احيةالتص يف .أخرا ،م
تق ية التصويت وفق اأغلبية مع ااست تاج العام ل ظام التص يف وتقوية اانسداد اةزئي.ال تائج التجريبية تشر
إ أن الكشف استخدام معاةة اللون و اإدرا اسسي للون البشري قادر على الكشف عن جسم اإنسان
مع ال سبة امئوية للكشف عن الوجه معدل ، ٪92والكشف عن اليد اليمى معدل ٪ 86و كشف اليد
اليسرى ايضا معدل .٪85وي الوقت نفسه ,أظهر أداء نظام الشبكة العصبية ان التص يف اجح ي حديد
الوجه ،اليد اليمى و اليد اليسرى وجاءت كاآي ٪73 ، ٪ 90و ٪ 74على التوا .ي حن ،تم العثور
على الدقة ةميع الطبقات التسعة من الفئة ِ( ِِأ) ح (ذ) لتكون ٪43واأعلى كانت .٪ 95ب اء على
نظام التص يف استخدام تق ية التصويت ،أظهرت ال تائج حس ا قليا على أداء التص يف ةميع الطبقات
التسعة حيث بلغت أدى زادة ٪ 45بي ماوصلت اأعل إ .٪100
iii
APPROVAL PAGE
I certify that I have supervised and read this study and that in my opinion; it conforms
to acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Master of Science in Mechatronics
Engineering.
….…..............................................
Yasir Mohd Mustafah
Supervisor
….…..............................................
Nahrul Khair Alang Md Rashid
Co-supervisor
I certify that I have read this study and that in my opinion it conforms to acceptable
standards of scholarly presentation and is fully adequate, in scope and quality, as a
thesis for the degree of Master of Science in Mechatronics Engineering.
……………………………………
Amir Akramin Shafie
Internal Examiner
……………………………………
Hadzli Hashim
External Examiner
This thesis was submitted to the Department of Mechatronics Engineering and it is
accepted as a fulfilment of the requirement for the degree of Master of Science in
Mechatronics Engineering.
……………………………………
Md Raisuddin Khan
Head, Department of
Mechatronics Engineering
This thesis was submitted to the Kulliyah of Engineering and is accepted as a
fulfilment of the requirements for the degree of Master of Science in Mechatronics
Engineering.
……………………………………
Md. Noor Salleh
Dean, Kulliyah of Engineering
iv
DECLARATION
I hereby declare that this dissertation is the result of my own investigations, except
where otherwise stated. I also declare that it has not been previously or concurrently
submitted as a whole for any other degrees of IIUM or other institutions.
Nurul Fatiha binti Johan
Signature …................................
Date ….........................
v
INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
DECLARATION OF COPYRIGHT AND AFFIRMATION
OF FAIR USE OF UNPUBLISHED RESEARCH
Copyright © 2015 by International Islamic University Malaysia. All rights reserved.
DETECTION OF PARTIALLY OCCLUDED HUMAN
USING SEPARATE BODY PARTS CLASSIFIER
I hereby affirm that The International Islamic University Malaysia (IIUM) holds all
rights in the copyright of this work and henceforth any reproduction or use in any
form or by means whatsoever is prohibited without the written consent of IIUM.
No part of this unpublished research may be reproduced, store in a retrieval system, or
transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise without prior written permission of the copyright holder.
Affirmed by Nurul Fatiha binti Johan
….......................................
Signature
….………
Date
vi
ACKNOWLEDGEMENTS
First of all, Alhamdulillah, a sincere praise to Allah the Almighty since with His
Power and Authorization, I have completed my master dissertation successfully. My
highest appreciation goes to Universiti Teknikal Malaysia Melaka (UTeM) for their
financial and moral support and gives me an extra time to complete this thesis.
A million of thank you to my supervisor, Dr Yasir Mohd Mustafah for his
suggested topic along with the encouragement and guidance during the research.
Thanks for his willingness to spend his time in giving ideas, instructions, support and
motivations throughout my research.
I am also thankful to my co-supervisor, Dr Nahrul Khair Alang Rashid for his
supervisions and keen support in assisting this work. I am deeply grateful to my
special friend, Nursabillilah bte Mohd Ali for many helpful suggestions and being
such a good tutor to me. Thanks for the encouragement and willingness to share the
ideas and information to complete this research. Next, the everest of thank you to my
family especially my husband for his support, patience, bless and understanding
during my study period.
Finally yet importantly, thanks to all my friends and lecturers whose direct and
indirect support helped me during this study. I really appreciate it and thank you for
what they have done for me.
vii
TABLE OF CONTENTS
Abstract……………………………………………………………………..….....
Abstract in Arabic…………………………………………………………………
Approval Page……………………………………………………………………..
Declaration Page……………………………………………………………..........
Copyright Page…………………………………………………………...…….….
Acknowledgements…………………………………………………...…………..
List of Tables…………………………………………………………………....…
List of Figures…………………………………………………………………......
List of Abbreviations…………………………………………………………....…
ii
iii
iv
v
vi
vii
xi
xiii
xv
CHAPTER ONE: INTRODUCTION………………………………………..
1.1 Introduction…………………………………………………………….
1.2 Problem Statement and Its Significant…………………………………
1.3 Research Objectives……………………………………………………
1.4 Research Scope………………………………………………………...
1.5 Research Methodology………………………………………………...
1.6 Dissertation Organization………………………………………………
1
1
3
4
4
5
8
CHAPTER TWO: LITERATURE REVIEW………………………………
2.1 Introduction…………………………………………………………….
2.2 Computer Vision and Image Processing……………………………….
2.2.1 Image Acquisition………………………………………………..
2.2.2 Color Image Processing…………………………………….........
2.2.2.1 RGB Color Space………………………………………...
2.2.2.2 HSI Color Space………………………………………….
2.2.2.3 YCbCr Color Space………………………………………
2.3 Related Works on Human Detection…………………………………..
2.4 Related Works on Skin Color of Human Body Detection……………..
2.5 Morphological Operation………………………………………………
2.6 Related Works on Classification Algorithm……………………………
2.7 Related Works on Artificial Neural Network…………………………..
2.8 Related Work on Voting Classification………………………………...
2.9 Related Works on Partial Occlusion……………………………………
3.0 Summary……………………………………………………………….
9
9
10
11
12
13
14
15
17
18
24
25
28
30
32
34
CHAPTER THREE: DEVELOPMENT OF HUMAN BODY PARTS
DETECTION SYSTEM……………………………...
3.1 Introduction…………………………………………………………….
3.2 Detection System Overview……………………………………………
3.3 Human Skin Color……………………………………………………..
3.4 Image Acquisition……………………………………………………...
35
viii
35
35
37
38
3.5 Skin Color Segmentation………………………………………………
3.5.1 YCbCr Color Space………………………………………............
3.5.2 RGB to YCbCr Conversion………………………………………
3.5.3 Skin Color Thresholding…………………………………………
3.5.4 Binary Formation………………………………………………...
3.6 Background Rejection………………………………………………….
3.6.1 Morphological Operation………………………………………...
3.6.1.1 Erosion and Dilation……………………………………..
3.6.1.2 Opening Operation……………………………………….
3.6.1.3 Closing Operation………………………………………..
3.7 Human Body Parts Detection System………………………………….
3.7.1 Bounding Box……………………………………………………
3.8 Results and Discussion…………………………………………………
3.8.1 Detection Result…………………………………………….........
3.8.2 Detection Accuracy………………………………………………
3.8.3 Detection Performance…………………………………………...
3.9 Summary……………………………………………………………….
39
39
40
41
43
44
44
45
46
47
48
49
51
51
54
55
57
CHAPTER FOUR: DEVELOPMENT OF HUMAN BODY PARTS
CLASSIFICATION SYSTEM…………………………
4.1 Introduction…………………………………………………………….
4.2 Pre-processing Data Acquisition……………………………………….
4.3 Dataset………………………………………………………………….
4.3.1 Feature Extraction………………………………………………..
4.4 Artificial Neural Network Intelligent System………………………….
4.4.1 Network Architecture and Learning Algorithm…………….........
4.4.2 Processing the Neurons…………………………………………..
4.4.3 Training and Testing of Artificial Neural Network………………
4.5 Face Classifier………………………………………………………….
4.5.1 System Designing……………………………………………….
4.5.2 Training of Face Classifier…………………………………........
4.5.3 Testing of Face Classifier………………………………………..
4.5.4 Recognition System of Face Classifier………………………….
4.5.5 Classification of Face Classifier………………………………...
4.6 Hands Classifier………………………………………………………..
4.6.1 System Designing………………………………………………..
4.6.1.1 Training of Right Hand Classifier……………………….
4.6.1.2 Training of Left Hand Classifier…………………...........
4.6.2 Recognition of Hands Classifier………………………………...
4.6.3 Classification of Hands Classifier……………………………….
4.7 Results and Discussion………………………………………………...
4.8 Summary………………………………………………………………
58
ix
58
58
60
61
62
62
63
65
67
68
70
71
74
75
77
78
79
80
80
82
83
86
CHAPTER FIVE: MAJORITY VOTING OF HUMAN BODY PARTS
CLASSIFIERS……………………………………………...
5.1 Introduction……………………………………………………………
5.2 Classification System Overview………………………………………
5.3 Voting of Classifier Overview…………………………………………
5.4 Majority Voting Rule (MVR) Technique……………………………...
5.5 Evaluation of Partially Occluded Human Body Parts…………………
5.6 Results and Discussion………………………………………………...
5.7.1 Accuracy of Human Classification Voting Result……………….
5.7 Summary………………………………………………………………
87
CHAPTER SIX: CONCLUSION AND RECOMMENDATION……………..
6.1 Conclusion…………………………………………………………….
6.2 Recommendation……………………………………………………...
100
100
102
87
87
88
89
94
96
96
99
REFERENCES…………………………………………………………………... 103
PUBLICATIONS………………………………………………………………… 109
APPENDIX A…………………………………………………………………….
x
110
LIST OF TABLES
Table No.
Page No.
2.1
Human detection methods using background subtraction
17
2.2
Human detection methods based on direct detection
18
2.3
Summary of Color Models
23
2.4
Types of classification algorithm and their performance rate
30
3.1
Skin color threshold value in YCbCr color space
42
3.2
Number of detection based on types of human detection result
53
3.3
Performance of human detection result
55
4.1
Successful classification of human body parts
84
5.1
Results from Class F
92
5.2
Sample of false Classification
93
5.3
Example of partial occlusion
94
5.4
Performance of class F using majority voting technique
97
5.5
Performance of the overall system
98
xi
LIST OF FIGURES
Figure No.
Page No
1.1
Flowchart of the research methodology
7
2.1
Digital image
12
2.2
RGB model in 3D
13
2.3
HSI color space
15
2.4
RGB color cube in the YCbCr space
16
2.5
Foreman image
22
2.6
The histogram distribution of Cb and Cr components
22
2.7
Skin color distribution
22
2.8
Erosion and dilation image
25
2.9
Opening and closing operation
25
3.1
Proposed system for human body parts detection
36
3.2
Skin color among various ethnics groups
37
3.3
Representation of pixel element
38
3.4
The separation of Y, Cb and Cr components
40
3.5
Conversion Image
41
3.6
Binary color formation
43
3.7
Structure of opening and closing operation system
45
3.8
Example of morphological basic operation
46
3.9
Opening operation
47
3.10
Closing Operation
47
3.11
Architecture of block diagram for human body parts detection
48
xii
3.12
Bounding box illustration
49
3.13
Human body parts detection image bounded in a rectangular box
50
3.14
True positive in human body parts detection
52
3.15
False positive in human body parts detection
52
3.16
False negative in detection of human body parts
53
3.17
The performance of detection result
56
4.1
Block diagram of the proposed system
58
4.2
The detected body parts before cropping
59
4.3
The cropped image
60
4.4
Distribution of human body dataset
60
4.5
Example of the face feature extraction technique
61
4.6
Feed-forward multilayer perceptron (MLP) network architecture
63
4.7
Artificial neurons structure
64
4.8
Generic representation of output and target data
66
4.9
Block diagram of classification system using ANN classifier
67
4.10
Face Images
68
4.11
Neural network architecture for face classifier
70
4.12
Training system
70
4.13
Testing system of neural network
72
4.14
Load testing file of 001 image
72
4.15
Image after load testing file
73
4.16
Load training data from 9 classes
73
4.17
The recognition testing process for human ID 001
74
4.18
The recognition testing process for human ID 037
75
4.19
Classification process
76
xiii
4.20
Hands image
78
4.21
Neural network architecture for hands classifier
79
4.22
Training system
79
4.23
Neural network architecture for left hand classifier and
its training system
80
4.24
The recognition testing result
81
4.25
Classification result
83
4.26
Processing speed for ANN recognition and classification
rate / frame
85
5.1
Classification system
88
5.2
Schematic diagram of proposed majority voting technique
for multiple classifiers system
89
5.3
Majority voting rule (MVR)
89
5.4
Human detection with right classification
91
5.5
Human detection with false classification
91
5.6
Image with partial occlusion
95
xiv
LIST OF ABBREVIATIONS
RGB
GVF
SVM
2D
3D
HSV
HSI
RBF
CP
LUT
ANN
MLP
MMLP
Tr
FPr
FNr
SCG
MVR
Red, green and blue
Gradient vector flow
Support vector machine
Two dimensional
Three dimensional
Hue, saturation and value
Hue, saturation and intensity
Radial basis function
Color predicate
Look-up table
Artificial neural network
Multilayer Perceptron
Multiple multilayer perceptron
True positive rate
False positive rate
False negative rate
Scale-conjugate gradient
Majority voting rule
xv
CHAPTER ONE
INTRODUCTION
1.1 INTRODUCTION
Currently, along with community development, public safety is a very important issue
that opens up various research topics including intelligent video surveillance to
increase public safety. Most of the buildings in metropolitan cities are using video
surveillance system in taking precautions especially in sensitive area. Moreover, these
days, even individuals are seeking for a security system not only for their own safety
but also for others such as detecting child abuse in kindergarten. Related research for
human detection and classification has thus given the priority in security, video
surveillance and privacy protection (Jadhav and Mane, 2009).
A robust method is needed in order to analyse the object of interest, to ensure
that the system can detect, recognize and classify the object of interest. Detection
means to detect or estimate the object of interest in the image while recognition is to
determine the similarity of object of interest to the reference object. Classification
basically is to classify an object of interest to specific category or class.
Detection and recognition of human in the images or video feeds are getting
more important nowadays with the aims to identify that the object in the image is
belong to human or not. There are several researches on automatics human detection
and classification algorithms have been done targeting numerous applications such for
video surveillance system, privacy protection, medical images analysis, information
security, behaviour analysis, tracking and search and rescue.
1
Detection of human is a complicated task due to the human appearance such as
clothing, articulation, shape of body and their pose and gesture. Although the position
of human like standing and walking has reduced the constraints of human gesture but
the possible variations are still large. In addition, the presence of multiple humans with
a moving camera and the human subjects may be switched to each other makes the
detection of human reliably becomes challenging task (Ru and Nevatia, 2006).
Since human detection from video is implemented on uncontrolled condition,
objects appear in the video are often affected by occlusion. Non-occlusion
environment still can be considered easy to implement in any system but how to cope
with partial occlusion is not yet reliable enough to solve. However, human body with
partial occlusion can be handled easily because if one of body part is occluded, the
human still can be detected using another body parts.
Although a lot of works have been done to improve existing human detection
system, there are still many issues arising among researchers to obtain a more
convenient system at a lower cost with less computation time and can be further
explore for the next research.
2
1.2 PROBLEM STATEMENT AND ITS SIGNIFICANCE
Human detection and classification is very important for various applications
especially in security and safety area. Hence, this topic has been extensively
researched. Currently, one of the main challenges on human detection is be able to
detect and label the human body parts in order to track the pose or gesture of an
individual. In addition, the human detection is always prone to occlusion by objects in
the scene or due to lighting. Human detection with partial occlusion can provide
solution for application such as search and rescue of a person in the crowded and
complex environment. The important aspect when dealing with an occluded human
body parts is human occlusion verification. The verification states whether the human
body parts is occluded or not and at the same time properly identify which part of
body is occluded. Generally, human body consists of various pose and shape.
Detection of a human can be done according to the structure of human body such as
head, face, neck, torso, limbs and etc. From literature reviewed, previous human
detection systems normally detect the full human body but sometimes lead to results in
failure when occlusion happens. However, none of the works focus on occlusion
detection but only on body parts tracking. Hence, the work is proposed in detecting
human by separate body parts classifiers which can detect human even when some
parts of the body is under occlusions. In addition, this detection system can also be
used to track human body parts for future research such as in pose and gesture
tracking.
3
1.3 RESEARCH OBJECTIVES
The objectives of this research are:
1. To design an algorithm to segment human body from complex background
scene using the fusion of skin colour detection algorithms.
2. To develop an intelligent human classification system using ANN
classifier based on feature extraction method.
3. To optimize the classification algorithm using majority voting of the body
parts classifiers suitable for classification under partial occlusion.
4. To evaluate the performance of the developed algorithm.
1.4 RESEARCH SCOPE
The research focuses on the development of detection and classification of human
body parts. The algorithm development is done using Matlab software. Digital camera
must be in a static position when capturing images in order for the system to work
properly. The developed system is able to detect and classify the human body by
extracting the features of human body parts individually. The human subject can be in
a various position for detection stage, however, for classification stage, the image
taken for the system database must be in frontal view with upright position. This
research considers the body parts that can be detected using human skin color feature
which cover only the face and hands. Since the skin color feature is used, it is
important that proper lighting is available. For partial occlusion, the system uses the
imaginary occlusion instead of using real occlusion.
4
1.5 RESEARCH METHODOLOGY
This research will be accomplished by developing and implementing the algorithm for
detection and classification of human body parts which includes the following research
stages:
1. Literature Review:
In order to make the research successful, advantage and disadvantage must
be taken as a guideline to create another work which is not exactly the
same, but could be much better than the original one. Information about the
previous works related to this research are collected and examined
thoroughly. Most of the source of the information is mainly obtained from
conference paper, journals, printed material and internet and then can be
proposed an approach that can be best to implement the required system.
2. Development of the Human Body Parts Detection Algorithm:
The main equipment for this research is computer and digital camera. The
purpose of using a computer is for the implementation of the algorithm and
a digital camera for evaluation and dataset construction. Skin color
technique is used to extract skin color features of the human. Then,
background rejection technique is implemented to eliminate noise or clutter
object in order to smooth the image for further processing. Modifications of
the existing technique are necessary to improve the efficiency of the
proposed system. In spite of this, it is worth to note that the main part of
this research is algorithm programming. Programming language or
software mode that is going to be employed throughout the processing
algorithm for detection and classification is based on Matlab 2012
software.
5
3. Development of the Human Body Parts Classification Algorithm:
System for classifying human is designed based on artificial intelligent
system of neural network with the aim to identify that either the extracted
body parts belongs to a particular class or not.
4. Development of Majority Voting of Human Body Parts Classifiers:
The voting technique is applied after the classification stage. From the
voting of multiple classifiers, the majority will determine the final result.
Apart from that, for partial occlusion, the proposed system is tested for
recognizing human subject under partial occlusion.
5. Evaluation of the System Performance:
The system performance is tested and analysed in terms of accuracy and
speed using the testing dataset.
6
Start
Literature Review
Image Acquisition
Development of Human Body
Parts Detection Algorithm
Development of Human Body
Parts Classification Algorithm
Development of Human Body
Parts Voting Classifier
Technique
Test and Evaluate the System
Satisfied?
No
Yes
End
Figure 1.1: Flowchart of research methodology
7
1.6 DISSERTATION ORGANIZATION
This dissertation consists of six chapters and organized as follows:
Chapter one firstly describes the introduction and background of study. Apart from
that, the problem statement, research objectives, research scopes and research
methodology also included in this chapter.
Chapter two covers the literature review on computer vision and images processing,
image acquisition, color image processing and the important parts on this chapter are
related work on human detection, skin color human body detection, morphological
operation, artificial neural network and voting classification technique.
Chapter three presents the concept of human detection in terms of human skin color,
image acquisition, skin color segmentation, background rejection method and lastly
detection of the human body parts.
Chapter four discusses on the classification starting from pre-processing of acquisition,
feature extraction and artificial neural network for classification.
Chapter five explains the voting technique and evaluation of the system of partial
occlusion problem.
Chapter six conclude this thesis and propose recommendation for future works.
8
USING SEPARATE BODY PARTS
CLASSIFIERS
BY
NURUL FATIHA BINTI JOHAN
INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
2015
© Universiti Teknikal Malaysia Melaka
DETECTION OF PARTIALLY OCCLUDED HUMAN
USING SEPARATE BODY PARTS
CLASSIFIERS
BY
NURUL FATIHA BINTI JOHAN
A dissertation submitted in fulfilment of the requirement
for the degree of Master of Science in Mechatronics
Engineering
Kulliyyah of Engineering
International Islamic University Malaysia
MAY 2015
© Universiti Teknikal Malaysia Melaka
ABSTRACT
The application of computer vision in the surveillance system has provided huge
advantages in the field of security and safety system. In recent years, human detection
and classification subjects have shown an increasing focus in finding specific
individual such as in the case of detecting person in crowded places at a time.
Detection and classification of human can be a challenging task due to the wide
variability of human appearance in terms of clothing, lighting conditions and the
occlusion. These constraints directly influence the effectiveness of the overall system.
To cope with these problems, human detection and classification system is presented
in this thesis which requires fast computations in addition of accurate results. The
propose system will first detect the human in an image by using YCbCr color
thresholding for skin color detection algorithm and then classify the body parts using
artificial intelligent neural network classifier into specific class and finally extend the
classification system with the majority voting technique in order to improve the
classification performance.The first hypothesis of the research is that YCbCr skin
color detection method can be used to detect and identify the exposed human body
parts even with the existence of various illumination conditions and complex
background. In this work, the body parts then only cover face and hands. The body
features are then extracted using feature extraction technique with the dimension of
region detected fixed to a standard size.These body features are then used as an input
to neural network system in order to classify the body parts into specific class.
Meanwhile each class consists of three classifier which is taken from the extracted
body regions and separated into face classifier, right hand classifier and left hand
classifier. Finally, the results of each body parts classification will be processed using
majority voting technique for overall conclusion of the classification system which is
robust to partial occlusion. Experimental results indicate that the human detection
using YCbCr color space is capable to detect the human body with the percentage of
face detection is 92%, right hand detection is 86% and left hand detection is 85%.
Meanwhile the performance of ANN classification system is successful in identifying
face, right hand and left hand which are 90%, 73% and 74% respectively. Whereas,
the accuracy of all 9 classes (Class A until Class I) is found to be 43% and highest to
be 95%. Based on the extended classification system using majority voting technique,
the results have shown a bit improvement on the classification performance for all 9
classes which is the lowest is increase to 45% and the highest is increase to 100%.
ii
خاصة البحث
إن التطبيقات عبالكمبيوتر ي نظام امراقبة وفرت مزاا هائلة ي جال أنظمة اأمن والسامة .ي الس وات
اأخرة ،أظهرت الكشف البشري واموضوعات اأخري أظهرت زادة ي القدرة علي تص يف الركيز ي
العثور على شخص معن ح لو كان ي اأماكن امزدمة ي الوقت نفسه .إن الكشف والتص يف البشري
مكن أن يكون مهمة صعبة نظرا للتقلبات الواسعة ي مظهر اإنسان من حيث امابس و ظروف اإضاءة و
انسداد الرؤا .هذ القيود تؤثر أثرا مباشرا على فعالية ال ظام العام.وابد من استخدام الكشف وتص يف
ال ظام البشري ي هذ اأطروحة ال تتطلب حساات سريعة اإضافة إ ال تائج الدقيقة .سيقوم ال ظام
الكشف أوا عن اإنسان ي شكل صورة استخدامعاةة اللون واإدرا اسسيمع خوارزمية الكشف ومن ثم
تصمف أجزاء اةسم استخدام الذكاء ااصط اعي وتص فهم لفئة معي ة ،وأخرا توسيع نظام التص يف مع تق ية
التصويت من أجل حسن اأداء ي التص يف وجعل امص ف أقوي من احية اانسداد اةزئي.ال ظام امقرح
يستخدم وفقخوارزمية الكشف عن لون البشرةوالشبكة العصبية ونظام الذكاء ااصط اعي حيث ب يت على
تسعةمواضع ي اةسمالبشري .والفرضية اأو من ي البحث اسا هي أن معاةة اللون واإدرا اسسي
طريقة للكشف عن لون البشرة مكن استخدامها للكشف و حديد أجزاء من جسم اإنسان ح مع وجود
الظروف امختلفة كاإضاءة واخلفية الغر واضحة .ي هذا البحث ،امهم من أجزاء جسم اإنسان الوجه
واليدين فقطومن ث يتم توضيخ اميزات استخدام تق ية التميز و البعد من م طقة اكتشاف ابتة إ حجم
يتم استخدام ميزات اةسم كمدخل ل ظام الشبكة العصبية من أجل تص يفججزاء اةسم إ فئات
قياسي .ث م
معي ة .إضافة ا ذلك ،يتكون كل ص ف من ثاثة أص اف اخذت من م اطق ي اةسم وص فت وفق اآي،
ستتم معاةة نتائج كل التصانيف أجزاء اةسم استخدام
اليد اليمى و اليسرى من احيةالتص يف .أخرا ،م
تق ية التصويت وفق اأغلبية مع ااست تاج العام ل ظام التص يف وتقوية اانسداد اةزئي.ال تائج التجريبية تشر
إ أن الكشف استخدام معاةة اللون و اإدرا اسسي للون البشري قادر على الكشف عن جسم اإنسان
مع ال سبة امئوية للكشف عن الوجه معدل ، ٪92والكشف عن اليد اليمى معدل ٪ 86و كشف اليد
اليسرى ايضا معدل .٪85وي الوقت نفسه ,أظهر أداء نظام الشبكة العصبية ان التص يف اجح ي حديد
الوجه ،اليد اليمى و اليد اليسرى وجاءت كاآي ٪73 ، ٪ 90و ٪ 74على التوا .ي حن ،تم العثور
على الدقة ةميع الطبقات التسعة من الفئة ِ( ِِأ) ح (ذ) لتكون ٪43واأعلى كانت .٪ 95ب اء على
نظام التص يف استخدام تق ية التصويت ،أظهرت ال تائج حس ا قليا على أداء التص يف ةميع الطبقات
التسعة حيث بلغت أدى زادة ٪ 45بي ماوصلت اأعل إ .٪100
iii
APPROVAL PAGE
I certify that I have supervised and read this study and that in my opinion; it conforms
to acceptable standards of scholarly presentation and is fully adequate, in scope and
quality, as a dissertation for the degree of Master of Science in Mechatronics
Engineering.
….…..............................................
Yasir Mohd Mustafah
Supervisor
….…..............................................
Nahrul Khair Alang Md Rashid
Co-supervisor
I certify that I have read this study and that in my opinion it conforms to acceptable
standards of scholarly presentation and is fully adequate, in scope and quality, as a
thesis for the degree of Master of Science in Mechatronics Engineering.
……………………………………
Amir Akramin Shafie
Internal Examiner
……………………………………
Hadzli Hashim
External Examiner
This thesis was submitted to the Department of Mechatronics Engineering and it is
accepted as a fulfilment of the requirement for the degree of Master of Science in
Mechatronics Engineering.
……………………………………
Md Raisuddin Khan
Head, Department of
Mechatronics Engineering
This thesis was submitted to the Kulliyah of Engineering and is accepted as a
fulfilment of the requirements for the degree of Master of Science in Mechatronics
Engineering.
……………………………………
Md. Noor Salleh
Dean, Kulliyah of Engineering
iv
DECLARATION
I hereby declare that this dissertation is the result of my own investigations, except
where otherwise stated. I also declare that it has not been previously or concurrently
submitted as a whole for any other degrees of IIUM or other institutions.
Nurul Fatiha binti Johan
Signature …................................
Date ….........................
v
INTERNATIONAL ISLAMIC UNIVERSITY MALAYSIA
DECLARATION OF COPYRIGHT AND AFFIRMATION
OF FAIR USE OF UNPUBLISHED RESEARCH
Copyright © 2015 by International Islamic University Malaysia. All rights reserved.
DETECTION OF PARTIALLY OCCLUDED HUMAN
USING SEPARATE BODY PARTS CLASSIFIER
I hereby affirm that The International Islamic University Malaysia (IIUM) holds all
rights in the copyright of this work and henceforth any reproduction or use in any
form or by means whatsoever is prohibited without the written consent of IIUM.
No part of this unpublished research may be reproduced, store in a retrieval system, or
transmitted, in any form or by any means, electronic, mechanical, photocopying,
recording or otherwise without prior written permission of the copyright holder.
Affirmed by Nurul Fatiha binti Johan
….......................................
Signature
….………
Date
vi
ACKNOWLEDGEMENTS
First of all, Alhamdulillah, a sincere praise to Allah the Almighty since with His
Power and Authorization, I have completed my master dissertation successfully. My
highest appreciation goes to Universiti Teknikal Malaysia Melaka (UTeM) for their
financial and moral support and gives me an extra time to complete this thesis.
A million of thank you to my supervisor, Dr Yasir Mohd Mustafah for his
suggested topic along with the encouragement and guidance during the research.
Thanks for his willingness to spend his time in giving ideas, instructions, support and
motivations throughout my research.
I am also thankful to my co-supervisor, Dr Nahrul Khair Alang Rashid for his
supervisions and keen support in assisting this work. I am deeply grateful to my
special friend, Nursabillilah bte Mohd Ali for many helpful suggestions and being
such a good tutor to me. Thanks for the encouragement and willingness to share the
ideas and information to complete this research. Next, the everest of thank you to my
family especially my husband for his support, patience, bless and understanding
during my study period.
Finally yet importantly, thanks to all my friends and lecturers whose direct and
indirect support helped me during this study. I really appreciate it and thank you for
what they have done for me.
vii
TABLE OF CONTENTS
Abstract……………………………………………………………………..….....
Abstract in Arabic…………………………………………………………………
Approval Page……………………………………………………………………..
Declaration Page……………………………………………………………..........
Copyright Page…………………………………………………………...…….….
Acknowledgements…………………………………………………...…………..
List of Tables…………………………………………………………………....…
List of Figures…………………………………………………………………......
List of Abbreviations…………………………………………………………....…
ii
iii
iv
v
vi
vii
xi
xiii
xv
CHAPTER ONE: INTRODUCTION………………………………………..
1.1 Introduction…………………………………………………………….
1.2 Problem Statement and Its Significant…………………………………
1.3 Research Objectives……………………………………………………
1.4 Research Scope………………………………………………………...
1.5 Research Methodology………………………………………………...
1.6 Dissertation Organization………………………………………………
1
1
3
4
4
5
8
CHAPTER TWO: LITERATURE REVIEW………………………………
2.1 Introduction…………………………………………………………….
2.2 Computer Vision and Image Processing……………………………….
2.2.1 Image Acquisition………………………………………………..
2.2.2 Color Image Processing…………………………………….........
2.2.2.1 RGB Color Space………………………………………...
2.2.2.2 HSI Color Space………………………………………….
2.2.2.3 YCbCr Color Space………………………………………
2.3 Related Works on Human Detection…………………………………..
2.4 Related Works on Skin Color of Human Body Detection……………..
2.5 Morphological Operation………………………………………………
2.6 Related Works on Classification Algorithm……………………………
2.7 Related Works on Artificial Neural Network…………………………..
2.8 Related Work on Voting Classification………………………………...
2.9 Related Works on Partial Occlusion……………………………………
3.0 Summary……………………………………………………………….
9
9
10
11
12
13
14
15
17
18
24
25
28
30
32
34
CHAPTER THREE: DEVELOPMENT OF HUMAN BODY PARTS
DETECTION SYSTEM……………………………...
3.1 Introduction…………………………………………………………….
3.2 Detection System Overview……………………………………………
3.3 Human Skin Color……………………………………………………..
3.4 Image Acquisition……………………………………………………...
35
viii
35
35
37
38
3.5 Skin Color Segmentation………………………………………………
3.5.1 YCbCr Color Space………………………………………............
3.5.2 RGB to YCbCr Conversion………………………………………
3.5.3 Skin Color Thresholding…………………………………………
3.5.4 Binary Formation………………………………………………...
3.6 Background Rejection………………………………………………….
3.6.1 Morphological Operation………………………………………...
3.6.1.1 Erosion and Dilation……………………………………..
3.6.1.2 Opening Operation……………………………………….
3.6.1.3 Closing Operation………………………………………..
3.7 Human Body Parts Detection System………………………………….
3.7.1 Bounding Box……………………………………………………
3.8 Results and Discussion…………………………………………………
3.8.1 Detection Result…………………………………………….........
3.8.2 Detection Accuracy………………………………………………
3.8.3 Detection Performance…………………………………………...
3.9 Summary……………………………………………………………….
39
39
40
41
43
44
44
45
46
47
48
49
51
51
54
55
57
CHAPTER FOUR: DEVELOPMENT OF HUMAN BODY PARTS
CLASSIFICATION SYSTEM…………………………
4.1 Introduction…………………………………………………………….
4.2 Pre-processing Data Acquisition……………………………………….
4.3 Dataset………………………………………………………………….
4.3.1 Feature Extraction………………………………………………..
4.4 Artificial Neural Network Intelligent System………………………….
4.4.1 Network Architecture and Learning Algorithm…………….........
4.4.2 Processing the Neurons…………………………………………..
4.4.3 Training and Testing of Artificial Neural Network………………
4.5 Face Classifier………………………………………………………….
4.5.1 System Designing……………………………………………….
4.5.2 Training of Face Classifier…………………………………........
4.5.3 Testing of Face Classifier………………………………………..
4.5.4 Recognition System of Face Classifier………………………….
4.5.5 Classification of Face Classifier………………………………...
4.6 Hands Classifier………………………………………………………..
4.6.1 System Designing………………………………………………..
4.6.1.1 Training of Right Hand Classifier……………………….
4.6.1.2 Training of Left Hand Classifier…………………...........
4.6.2 Recognition of Hands Classifier………………………………...
4.6.3 Classification of Hands Classifier……………………………….
4.7 Results and Discussion………………………………………………...
4.8 Summary………………………………………………………………
58
ix
58
58
60
61
62
62
63
65
67
68
70
71
74
75
77
78
79
80
80
82
83
86
CHAPTER FIVE: MAJORITY VOTING OF HUMAN BODY PARTS
CLASSIFIERS……………………………………………...
5.1 Introduction……………………………………………………………
5.2 Classification System Overview………………………………………
5.3 Voting of Classifier Overview…………………………………………
5.4 Majority Voting Rule (MVR) Technique……………………………...
5.5 Evaluation of Partially Occluded Human Body Parts…………………
5.6 Results and Discussion………………………………………………...
5.7.1 Accuracy of Human Classification Voting Result……………….
5.7 Summary………………………………………………………………
87
CHAPTER SIX: CONCLUSION AND RECOMMENDATION……………..
6.1 Conclusion…………………………………………………………….
6.2 Recommendation……………………………………………………...
100
100
102
87
87
88
89
94
96
96
99
REFERENCES…………………………………………………………………... 103
PUBLICATIONS………………………………………………………………… 109
APPENDIX A…………………………………………………………………….
x
110
LIST OF TABLES
Table No.
Page No.
2.1
Human detection methods using background subtraction
17
2.2
Human detection methods based on direct detection
18
2.3
Summary of Color Models
23
2.4
Types of classification algorithm and their performance rate
30
3.1
Skin color threshold value in YCbCr color space
42
3.2
Number of detection based on types of human detection result
53
3.3
Performance of human detection result
55
4.1
Successful classification of human body parts
84
5.1
Results from Class F
92
5.2
Sample of false Classification
93
5.3
Example of partial occlusion
94
5.4
Performance of class F using majority voting technique
97
5.5
Performance of the overall system
98
xi
LIST OF FIGURES
Figure No.
Page No
1.1
Flowchart of the research methodology
7
2.1
Digital image
12
2.2
RGB model in 3D
13
2.3
HSI color space
15
2.4
RGB color cube in the YCbCr space
16
2.5
Foreman image
22
2.6
The histogram distribution of Cb and Cr components
22
2.7
Skin color distribution
22
2.8
Erosion and dilation image
25
2.9
Opening and closing operation
25
3.1
Proposed system for human body parts detection
36
3.2
Skin color among various ethnics groups
37
3.3
Representation of pixel element
38
3.4
The separation of Y, Cb and Cr components
40
3.5
Conversion Image
41
3.6
Binary color formation
43
3.7
Structure of opening and closing operation system
45
3.8
Example of morphological basic operation
46
3.9
Opening operation
47
3.10
Closing Operation
47
3.11
Architecture of block diagram for human body parts detection
48
xii
3.12
Bounding box illustration
49
3.13
Human body parts detection image bounded in a rectangular box
50
3.14
True positive in human body parts detection
52
3.15
False positive in human body parts detection
52
3.16
False negative in detection of human body parts
53
3.17
The performance of detection result
56
4.1
Block diagram of the proposed system
58
4.2
The detected body parts before cropping
59
4.3
The cropped image
60
4.4
Distribution of human body dataset
60
4.5
Example of the face feature extraction technique
61
4.6
Feed-forward multilayer perceptron (MLP) network architecture
63
4.7
Artificial neurons structure
64
4.8
Generic representation of output and target data
66
4.9
Block diagram of classification system using ANN classifier
67
4.10
Face Images
68
4.11
Neural network architecture for face classifier
70
4.12
Training system
70
4.13
Testing system of neural network
72
4.14
Load testing file of 001 image
72
4.15
Image after load testing file
73
4.16
Load training data from 9 classes
73
4.17
The recognition testing process for human ID 001
74
4.18
The recognition testing process for human ID 037
75
4.19
Classification process
76
xiii
4.20
Hands image
78
4.21
Neural network architecture for hands classifier
79
4.22
Training system
79
4.23
Neural network architecture for left hand classifier and
its training system
80
4.24
The recognition testing result
81
4.25
Classification result
83
4.26
Processing speed for ANN recognition and classification
rate / frame
85
5.1
Classification system
88
5.2
Schematic diagram of proposed majority voting technique
for multiple classifiers system
89
5.3
Majority voting rule (MVR)
89
5.4
Human detection with right classification
91
5.5
Human detection with false classification
91
5.6
Image with partial occlusion
95
xiv
LIST OF ABBREVIATIONS
RGB
GVF
SVM
2D
3D
HSV
HSI
RBF
CP
LUT
ANN
MLP
MMLP
Tr
FPr
FNr
SCG
MVR
Red, green and blue
Gradient vector flow
Support vector machine
Two dimensional
Three dimensional
Hue, saturation and value
Hue, saturation and intensity
Radial basis function
Color predicate
Look-up table
Artificial neural network
Multilayer Perceptron
Multiple multilayer perceptron
True positive rate
False positive rate
False negative rate
Scale-conjugate gradient
Majority voting rule
xv
CHAPTER ONE
INTRODUCTION
1.1 INTRODUCTION
Currently, along with community development, public safety is a very important issue
that opens up various research topics including intelligent video surveillance to
increase public safety. Most of the buildings in metropolitan cities are using video
surveillance system in taking precautions especially in sensitive area. Moreover, these
days, even individuals are seeking for a security system not only for their own safety
but also for others such as detecting child abuse in kindergarten. Related research for
human detection and classification has thus given the priority in security, video
surveillance and privacy protection (Jadhav and Mane, 2009).
A robust method is needed in order to analyse the object of interest, to ensure
that the system can detect, recognize and classify the object of interest. Detection
means to detect or estimate the object of interest in the image while recognition is to
determine the similarity of object of interest to the reference object. Classification
basically is to classify an object of interest to specific category or class.
Detection and recognition of human in the images or video feeds are getting
more important nowadays with the aims to identify that the object in the image is
belong to human or not. There are several researches on automatics human detection
and classification algorithms have been done targeting numerous applications such for
video surveillance system, privacy protection, medical images analysis, information
security, behaviour analysis, tracking and search and rescue.
1
Detection of human is a complicated task due to the human appearance such as
clothing, articulation, shape of body and their pose and gesture. Although the position
of human like standing and walking has reduced the constraints of human gesture but
the possible variations are still large. In addition, the presence of multiple humans with
a moving camera and the human subjects may be switched to each other makes the
detection of human reliably becomes challenging task (Ru and Nevatia, 2006).
Since human detection from video is implemented on uncontrolled condition,
objects appear in the video are often affected by occlusion. Non-occlusion
environment still can be considered easy to implement in any system but how to cope
with partial occlusion is not yet reliable enough to solve. However, human body with
partial occlusion can be handled easily because if one of body part is occluded, the
human still can be detected using another body parts.
Although a lot of works have been done to improve existing human detection
system, there are still many issues arising among researchers to obtain a more
convenient system at a lower cost with less computation time and can be further
explore for the next research.
2
1.2 PROBLEM STATEMENT AND ITS SIGNIFICANCE
Human detection and classification is very important for various applications
especially in security and safety area. Hence, this topic has been extensively
researched. Currently, one of the main challenges on human detection is be able to
detect and label the human body parts in order to track the pose or gesture of an
individual. In addition, the human detection is always prone to occlusion by objects in
the scene or due to lighting. Human detection with partial occlusion can provide
solution for application such as search and rescue of a person in the crowded and
complex environment. The important aspect when dealing with an occluded human
body parts is human occlusion verification. The verification states whether the human
body parts is occluded or not and at the same time properly identify which part of
body is occluded. Generally, human body consists of various pose and shape.
Detection of a human can be done according to the structure of human body such as
head, face, neck, torso, limbs and etc. From literature reviewed, previous human
detection systems normally detect the full human body but sometimes lead to results in
failure when occlusion happens. However, none of the works focus on occlusion
detection but only on body parts tracking. Hence, the work is proposed in detecting
human by separate body parts classifiers which can detect human even when some
parts of the body is under occlusions. In addition, this detection system can also be
used to track human body parts for future research such as in pose and gesture
tracking.
3
1.3 RESEARCH OBJECTIVES
The objectives of this research are:
1. To design an algorithm to segment human body from complex background
scene using the fusion of skin colour detection algorithms.
2. To develop an intelligent human classification system using ANN
classifier based on feature extraction method.
3. To optimize the classification algorithm using majority voting of the body
parts classifiers suitable for classification under partial occlusion.
4. To evaluate the performance of the developed algorithm.
1.4 RESEARCH SCOPE
The research focuses on the development of detection and classification of human
body parts. The algorithm development is done using Matlab software. Digital camera
must be in a static position when capturing images in order for the system to work
properly. The developed system is able to detect and classify the human body by
extracting the features of human body parts individually. The human subject can be in
a various position for detection stage, however, for classification stage, the image
taken for the system database must be in frontal view with upright position. This
research considers the body parts that can be detected using human skin color feature
which cover only the face and hands. Since the skin color feature is used, it is
important that proper lighting is available. For partial occlusion, the system uses the
imaginary occlusion instead of using real occlusion.
4
1.5 RESEARCH METHODOLOGY
This research will be accomplished by developing and implementing the algorithm for
detection and classification of human body parts which includes the following research
stages:
1. Literature Review:
In order to make the research successful, advantage and disadvantage must
be taken as a guideline to create another work which is not exactly the
same, but could be much better than the original one. Information about the
previous works related to this research are collected and examined
thoroughly. Most of the source of the information is mainly obtained from
conference paper, journals, printed material and internet and then can be
proposed an approach that can be best to implement the required system.
2. Development of the Human Body Parts Detection Algorithm:
The main equipment for this research is computer and digital camera. The
purpose of using a computer is for the implementation of the algorithm and
a digital camera for evaluation and dataset construction. Skin color
technique is used to extract skin color features of the human. Then,
background rejection technique is implemented to eliminate noise or clutter
object in order to smooth the image for further processing. Modifications of
the existing technique are necessary to improve the efficiency of the
proposed system. In spite of this, it is worth to note that the main part of
this research is algorithm programming. Programming language or
software mode that is going to be employed throughout the processing
algorithm for detection and classification is based on Matlab 2012
software.
5
3. Development of the Human Body Parts Classification Algorithm:
System for classifying human is designed based on artificial intelligent
system of neural network with the aim to identify that either the extracted
body parts belongs to a particular class or not.
4. Development of Majority Voting of Human Body Parts Classifiers:
The voting technique is applied after the classification stage. From the
voting of multiple classifiers, the majority will determine the final result.
Apart from that, for partial occlusion, the proposed system is tested for
recognizing human subject under partial occlusion.
5. Evaluation of the System Performance:
The system performance is tested and analysed in terms of accuracy and
speed using the testing dataset.
6
Start
Literature Review
Image Acquisition
Development of Human Body
Parts Detection Algorithm
Development of Human Body
Parts Classification Algorithm
Development of Human Body
Parts Voting Classifier
Technique
Test and Evaluate the System
Satisfied?
No
Yes
End
Figure 1.1: Flowchart of research methodology
7
1.6 DISSERTATION ORGANIZATION
This dissertation consists of six chapters and organized as follows:
Chapter one firstly describes the introduction and background of study. Apart from
that, the problem statement, research objectives, research scopes and research
methodology also included in this chapter.
Chapter two covers the literature review on computer vision and images processing,
image acquisition, color image processing and the important parts on this chapter are
related work on human detection, skin color human body detection, morphological
operation, artificial neural network and voting classification technique.
Chapter three presents the concept of human detection in terms of human skin color,
image acquisition, skin color segmentation, background rejection method and lastly
detection of the human body parts.
Chapter four discusses on the classification starting from pre-processing of acquisition,
feature extraction and artificial neural network for classification.
Chapter five explains the voting technique and evaluation of the system of partial
occlusion problem.
Chapter six conclude this thesis and propose recommendation for future works.
8