Convolutional neural networks for face recognition and finger-vein biometric identification.
CONVOLUTIONAL NEURAL NETWORKS FOR FACE RECOGNITION AND
FINGER-VEIN BIOMETRIC IDENTIFICATION
SYAFEEZA AHMAD RADZI
UNIVERSITI TEKNOLOGI MALAYSIA
© Universiti Teknikal Malaysia Melaka
PSZ 19:16 (Pind. 1/07)
UNIVERSITI TEKNOLOGI MALAYSIA
DECLARATION OF THESIS / UNDERGRADUATE PROJECT PAPER AND COPYRIGHT
Author’s full name :
Date of birth
:
Title
:
SYAFEEZA BINTI AHMAD RADZI
24TH MARCH 1981
CONVOLUTIONAL NEURAL NETWORKS FOR FACE RECOGNITION
AND FINGER-VEIN BIOMETRIC IDENTIFICATION
Academic Session:
2013/2014/2
I declare that this thesis is classified as :
√
CONFIDENTIAL
(Contains confidential information under the Official Secret
Act 1972)*
RESTRICTED
(Contains restricted information as specified by the
organisation where research was done)*
OPEN ACCESS
I agree that my thesis to be published as online open access
(full text)
I acknowledged that Universiti Teknologi Malaysia reserves the right as follows :
1. The thesis is the property of Universiti Teknologi Malaysia.
2. The Library of Universiti Teknologi Malaysia has the right to make copies for the purpose
of research only.
3. The Library has the right to make copies of the thesis for academic exchange.
Certified by :
SIGNATURE
810324-71-5024
SIGNATURE OF SUPERVISOR
MOHAMED KHALIL BIN MOHD HANI
(NEW IC NO. /PASSPORT NO.)
NAME OF SUPERVISOR
Date : 2 JULY 2014
NOTES :
*
Date : 2 JULY 2014
If the thesis is CONFIDENTIAL or RESTRICTED, please attach with the letter from
the organisation with period and reasons for confidentiality or restriction.
“I hereby declare that I have read this thesis and in my
opinion this thesis is sufficient in terms of scope and quality for the
award of the degree of Doctor of Philosophy (Electrical Engineering)”
Signature
:
Name
:
Prof. Dr. Mohamed Khalil Mohd. Hani
Date
:
July 2, 2014
BAHAGIAN A - Pengesahan Kerjasama*
Adalah disahkan bahawa projek penyelidikan tesis ini telah dilaksanakan melalui kerjasama
antara _ _ _ _ _ _ _ _ _ _ dengan _ _ _ _ _ __ _ __
Disahkan oleh:
Tandatangan
Tarikh:
Nama
Jawatan
(Cop rasmi)
* Jika penyediaan tesislprojek melibatkan kerjasama.
BAHAGIAN B- Untuk Kegunaan Pejabat Sekolah Pengajian Siswazah
Tesis ini telah diperiksa dan diakui oleh:
Nama dan Alamat Pemeriksa Luar
Prof. Dr. Mohd Alauddin Mohd Ali
Department of Electrical Electronic & Systems
Engineering,
Faculty of Engineering and Built Environment,
Universiti Kebangsaan Malaysia,
43600 Bangi, Se!ang-0>r.
Nama dan Alamat Pemeriksa Dalam
Prof. Dr. Jasmy bin Yunus
Dekan
Fakulti Biosains dan Kejuruteraan Perubatan,
UTM Johor Bahru.
Disahkan oleh Timbalan Pendaftar di Sekolah Pengajian Siswazah:
Tandatangan :
Nama
Tarikh :
ASRAM BIN SULAIMAN @ SAIM
© Universiti Teknikal Malaysia Melaka
CONVOLUTIONAL NEURAL NETWORKS FOR FACE RECOGNITION AND
FINGER-VEIN BIOMETRIC IDENTIFICATION
SYAFEEZA AHMAD RADZI
A thesis submitted in fulfilment of the
requirements for the award of the degree of
Doctor of Philosophy (Electrical Engineering)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
JULY 2014
© Universiti Teknikal Malaysia Melaka
ii
I declare that this thesis entitled "Convolutional Neural Networks for Face Recognition
and Finger-vein Biometric Identification" is the result of my own research except as
cited in the references. The thesis has not been accepted for any degree and is not
concurrently submitted in candidature of any other degree.
Signature
Name
SYAFEEZA AHMAD RADZI
Date
July 2, 2014
© Universiti Teknikal Malaysia Melaka
iii
Dedicated to my beloved parents, husband and daughter.
iv
ACKNOWLEDGEMENT
I am very blessed with the completion of this thesis. With this opportunity,
I would like to express my gratitude to several people that have involved directly or
indirectly to the completion of this thesis.
Firstly, I would like to express my sincere appreciation to my supervisor
Professor Dr. Mohamed Khalil bin Mohd Hani for the useful critics, guidance and
advices during the journey. I have gained valuable knowledge during his supervision.
It was also a privilege to work closely with the members of VeCAD Lab namely Rabia,
Vishnu, Lee Yee Hui, Sia Chen Wei, Moganesh and Liew Shan Sung. I am very
fortunate to have these people during my ups and downs of conducting this research. I
am glad for the sharing of constructive ideas and their concern.
Most importantly, I would like to thank my family especially my parents,
husband and daughter. Thank you for the continuous encouragement, patience and
boundless support in all my endeavours.
Last but not least, I would like to acknowledge Universiti Teknikal Malaysia
Melaka (UTeM) for giving me the opportunity to pursue my study up to this level.
v
ABSTRACT
The Convolutional Neural Network (CNN), a variant of the Multilayer
Perceptron (MLP), has shown promise in solving complex recognition problems,
particularly in visual pattern recognition. However, the classical LeNet-5 CNN model,
which most solutions are based on, is highly compute-intensive. This CNN also suffers
from long training time, due to the large number of layers that ranges from six to
eight. In this research, a CNN model with a reduced complexity is proposed for
application in face recognition and finger-vein biometric identification. A simpler
architecture is obtained by fusing convolutional and subsampling layers into one
layer, in conjunction with a partial connection scheme applied between the first two
layers in the network. As a result, the total number of layers is reduced to four.
The number of feature maps at each layer is optimized according to the type of
image database being processed. Consequently, the numbers of network parameters
(including neurons, trainable parameters and connections) are significantly reduced,
essentially increasing the generalization ability of the network. The Stochastic
Diagonal Levenberg-Marquadt (SDLM) backpropagation algorithm is modified and
applied in the training of the proposed network. With this learning algorithm, the
convergence rate is accelerated such that the proposed CNN converges within 15
epochs. For face recognition, the proposed CNN achieves recognition rates of 100.00%
and 99.50% for AT&T and AR Purdue face databases respectively. Recognition time
on the AT&T database is less than 0.003 seconds. These results outperform previous
existing works. In addition, when compared with the other CNN-based face recognizer,
the proposed CNN model has the least number of network parameters, hence better
generalization ability. A training scheme is also proposed to recognize new categories
without full CNN training. In this research, a novel CNN solution for the finger-vein
biometric identification problem is also proposed. To the best of knowledge, there is
no previous work reported in literature that applied CNN for finger-vein recognition.
The proposed method is efficient in that simple preprocessing algorithms are deployed.
The CNN design is adapted on a finger-vein database, which is developed in-house and
contains 81 subjects. A recognition accuracy of 99.38% is achieved, which is similar
to the results of state-of-the-art work. In conclusion, the success of the research in
solving face recognition and finger-vein biometric identification problems proves the
feasibility of the proposed CNN model in any pattern recognition system.
vi
ABSTRAK
Convolutional Neural Network (CNN) yang merupakan variasi kepada
Multilayer Perceptron (MLP) telah menunjukkan kebolehan dalam kerja pengecaman
yang rumit terutamanya dalam pengecaman corak visual. Walau bagaimanapun,
senibina klasik CNN iaitu LeNet-5, yang merupakan asas kepada kebanyakan
penyelesaian, mempunyai pengiraan intensif yang tinggi. CNN ini juga berhadapan
dengan masa latihan yang terlalu lama disebabkan oleh bilangan lapisannya dalam
lingkungan enam hingga lapan lapisan. Dalam kajian ini, model CNN dengan
kurang kekompleksan telah dicadangkan untuk diaplikasi pada pengecaman muka dan
pengesahan identiti biometrik urat jari. Senibina yang lebih ringkas telah diperolehi
dengan cara mencantumkan lapisan convolution dan subsampling ke satu lapisan,
dengan gabungan skim sambungan separa antara dua lapisan pertama dalam rangkaian.
Keputusannya, jumlah lapisan telah dikurangkan kepada empat. Bilangan petak sifat
pada setiap lapisan telah dioptimumkan berdasarkan kepada jenis pangkalan data yang
digunakan. Kesannya, bilangan parameter rangkaian (termasuk neuron, parameter
terlatih dan sambungan) nyata sekali dapat dikurangkan, terutamanya kebolehan
generalisasi yang lebih baik. Algoritma Stochastic Diagonal Levenberg-Marquadt
(SDLM) telah diubah suai dan diaplikasi dalam latihan rangkaian yang dicadangkan.
Dengan algoritma ini, kadar pembelajaran titik tumpu telah dipercepatkan untuk tumpu
dalam tempoh 15 epoch. Untuk pengecaman muka, CNN yang dicadangkan mencapai
kadar pengecaman sebanyak 100.00% dan 99.50% masing-masing untuk pangkalan
data AT&T dan AR Purdue. Masa pengecaman untuk AT&T adalah kurang daripada
0.003 saat. Keputusan yang diperolehi telah mengatasi kerja terdahulu. Tambahan
pula, apabila dibandingkan dengan reka bentuk CNN yang lain, senibina CNN yang
diusulkan mempunyai parameter rangkaian yang paling sedikit malahan ia mempunyai
kebolehan generalisasi yang lebih baik. Satu skim latihan juga telah dicadangkan untuk
mengecam kategori baru tanpa memerlukan keseluruhan latihan CNN. Dalam kajian
ini, penyelesaian CNN untuk masalah pengesahan identiti biometrik urat jari juga
telah dicadangkan. Sepanjang pengetahuan yang ada, tiada kerja sebelumnya yang
dilaporkan mengaplikasi CNN untuk pengesahan identiti biometrik urat jari. Kaedah
yang dicadangkan berkesan kerana algoritma pemprosesan mudah digunakan. Reka
bentuk CNN diadaptasi pada pangkalan data urat jari, yang telah dihasilkan sendiri dan
mengandungi 81 orang. Kejituan pengecaman sebanyak 99.38% telah dicapai, yang
hampir sama dengan keputusan yang diperoleh daripada kerja terkini. Kesimpulannya,
kajian ini telah berjaya menyelesaikan masalah pengecaman muka dan pengesahan
identiti biometrik urat jari membuktikan bahawa CNN yang dicadangkan boleh
dilaksanakan dalam sebarang sistem pengecaman corak.
vii
TABLE OF CONTENTS
CHAPTER
TITLE
DECLARATION
DEDICATION
ACKNOWLEDGEMENT
ABSTRACT
ABSTRAK
TABLE OF CONTENTS
LIST OF TABLES
LIST OF FIGURES
LIST OF ABBREVIATIONS
1
2
INTRODUCTION
1.1
Overview of Pattern Recognition
1.2
Neural Networks in Pattern Recognition Problems
1.3
Convolutional Neural Network
1.4
Biometric Pattern Recognition
1.4.1
Summary of Existing Face Recognizers
1.4.2
Summary of Work on Finger-vein Pattern
Recognition
1.5
Problem Statement
1.6
Research Objectives
1.7
Scope of Work
1.8
Research Contributions
1.9
Thesis Organization
LITERATURE REVIEW
2.1
Basic CNN Architectures
2.2
Previous Work applying CNN in Face Recognition
2.3
Related Previous Work on Finger-vein Biometric
Identification
PAGE
ii
iii
iv
v
vi
vii
xii
xv
xviii
1
1
2
3
4
7
8
8
13
14
14
16
17
17
19
23
viii
2.4
2.5
3
4
Other State-of-the-Art CNN Architectures
2.4.1
Siamese CNN
2.4.2
Space Displacement Neural Networks
2.4.3
Shunting Inhibitory Convolutional Neural
Networks
2.4.4
Sparse Convolutional Neural Networks
2.4.5
Multiscale Convolutional Neural Networks
2.4.6
Multiple Circular Path Convolutional
Neural Networks
2.4.7
Multi-column Deep Neural Networks
2.4.8
3D Convolutional Neural Networks
2.4.9
Recurrent Convolutional Neural Networks
Summary
CONVOLUTIONAL NEURAL NETWORK - ALGORITHMS AND ARCHITECTURE
3.1
Multilayer Perceptron Neural Network (MLPNN)
3.1.1
Activation Functions
3.1.2
Weight Initialization Methods
3.1.3
Normalization Method
3.2
Neural Network Training
3.2.1
Gradient Descent Backpropagation Algorithm
3.2.2
Resilient Backpropagation
3.3
LeNet-5 Convolutional Neural Networks
3.3.1
Algorithms in LeNet-5
3.3.2
Convolutional Neural Network Training
3.4
Summary
PROPOSED
CONVOLUTIONAL
NEURAL
NETWORK: MODELLING, ARCHITECTURE AND
ALGORITHM
4.1
Methodology
4.1.1
Cross-validation Technique to optimize
Generalization Ability
4.1.2
Techniques to Improve Convergence Rate
27
27
31
32
34
34
35
36
37
39
39
40
40
42
42
44
45
46
49
49
50
54
55
57
57
57
59
ix
4.2
4.3
4.4
4.5
4.6
5
4.1.3
Winner-Takes-All Rule
Proposed CNN Model
4.2.1
Introduction
4.2.2
CNN with Fused Convolution/Subsampling
4.2.3
Connection Schemes
Proposed Training Algorithm
4.3.1
Stochastic
Diagonal
LevenbergMarquardt (SDLM)
4.3.2
Proposed Modifications to SDLM Algorithm
Proposed Training Scheme to recognize New
Categories without Full CNN Training
Generalizing the CNN Architecture for New
Problem Domain
Summary
EXPERIMENTAL WORK AND RESULTS FOR CNN IN
FACE RECOGNITION
5.1
Performance Evaluation Procedures
5.2
Data Preparation
5.2.1
Face databases
5.3
Experimental Work and Results of the Proposed
Baseline CNN Model
5.3.1
Results of Cross-validation
5.3.2
Best Connection Scheme
5.3.3
The Most Suitable Normalization and
Weight Initialization Methods
5.3.4
The Optimum Input Image Size
5.3.5
Benchmarking Results
5.3.5.1 Benchmarking of Accuracy
5.3.5.2 Benchmarking of Execution
and Convergence Time
5.3.5.3 Analysis and Benchmarking of
Model Complexity
5.3.5.4 Comparison between the proposed CNN Model and LeNet-5
5.4
Experimental Work and Results of Proposed CNN
Model
62
63
63
63
67
69
69
71
73
76
77
78
78
82
82
85
86
87
88
89
90
90
92
93
95
96
x
5.4.1
5.4.2
5.4.3
5.5
5.6
5.7
5.8
6
Results of Cross-validation
Best Connection Scheme
The Most Suitable Normalization and
Weight Initialization Methods
5.4.4
The Optimum Input Image Size
5.4.5
Benchmarking Results
Experimental Results of CNN Model Targeted for
Face Images of the ”Extreme Challenge” Type
5.5.1
Results of Cross-validation
5.5.2
Best Connection Scheme
5.5.3
The Most Suitable Normalization and
Weight Initialization Methods
5.5.4
The Optimum Input Image Size
5.5.5
Benchmarking Results
Analysis of the Proposed Modification on SDLM
Algorithm
5.6.1
Comparison between the Proposed Modification on SDLM Algorithm and Standard Backpropagation
Analysis of Training Scheme to recognize New
Categories without Full CNN Training
Conclusion
EXPERIMENTAL WORK AND RESULTS FOR CNN IN
FINGER-VEIN BIOMETRIC IDENTIFICATION
6.1
Generation of Finger-vein Database for Biometric
Identification
6.2
Performance Evaluation of Finger-vein Biometric
Identification
6.2.1
VeCAD-UTM database
6.2.2
Results of Cross-validation
6.2.3
Best Connection Scheme
6.2.4
The Most Suitable Normalization and
Weight Initialization Methods
6.2.5
The Optimum Input Image Size
6.2.6
Benchmarking Results
6.3
Conclusion
96
96
97
99
100
101
101
103
104
105
107
107
109
111
112
113
113
118
118
118
120
120
123
124
126
xi
7
CONCLUSION
7.1
Concluding Remarks
7.2
Suggestions for Future Work
127
127
130
REFERENCES
132
Appendix A
146
xii
LIST OF TABLES
TABLE NO.
1.1
1.2
1.3
1.4
1.5
1.6
2.1
2.2
2.3
2.4
2.5
3.1
3.2
4.1
4.2
4.3
4.4
4.5
5.1
5.2
5.3
TITLE
Characteristic comparison of biometric application [1]
History of some biometric development [2]
Previous work on face recognition based on CNN
Previous work on face recognition applying non-CNN
methods
Accuracy achieved by Computational Intelligence (CI)
approach for finger-vein identification system
Detail information and accuracy achieved by conventional
approach for fingervein identification system
Face Recognition System using CNN
Conventional methods of finger-vein identification
Algorithms applied by CI approach for finger-vein identification system
Accuracy achieved by CI-based for finger-vein identification
Related previous work on finger-vein biometric system and
its recognition speed
Input/output relation and symbol for each activation function
Each row indicates which feature maps in S2 are combined
by the particular feature maps of C3 [3]
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for AT&T
database
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for AR Purdue
database
The values of ηg as given in [3]
Pose variations of FERET database
Performance of the model tested for AT&T database
Variant type of connections between C1 and C2 layer
PAGE
6
6
7
7
8
8
22
25
26
27
27
43
53
68
68
69
69
73
85
87
87
xiii
5.4
5.5
5.6
5.7
5.8
5.9
5.10
5.11
5.12
5.13
5.14
5.15
5.16
5.17
5.18
5.19
5.20
5.21
5.22
5.23
5.24
5.25
5.26
5.27
5.28
5.29
Connection scheme between C1 and C2 layer for AT&T
database
Parameter Setup for AT&T database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for AT&T database
Accuracy of each input size for AT&T database
Benchmarking of accuracy of other CNN face recognizers on
AT&T database
Benchmarking with non-CNN approach
Benchmarking of execution time and convergence learning
time against other face recognizers
Benchmarking of model complexity against previous work
that apply CNN (not restricted to face recognition problems)
Comparison between fused architecture and LeNet-5
Performance of the model tested for AR database
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for AR Purdue
database
Parameter Setup for AR Purdue database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for AR Purdue database
Parameter setup for each input size for AR Purdue database
Accuracy of each input size for AR Purdue database
Benchmarking of face identification system using AR Purdue
database with existing works
Performance of the model tested for FERET database
Variant type of connections between C1 and C2 layer for
FERET database
Connection scheme between C1 and C2 layer for FERET
database
Parameter Setup for FERET database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for FERET database
Parameter setup for each input size for FERET database
Accuracy of each input size for FERET database
Benchmarking of face identification system using FERET
database with existing works
Experimental setup for standard Backpropagation (BP)
88
88
88
89
91
91
92
95
96
97
98
98
99
99
100
100
102
103
103
104
104
105
105
105
107
110
xiv
5.30
5.31
5.32
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
6.11
Result of BP training using combination of different initial
learning rate and learning rate fading
Result of BP training using combination of different initial
learning rate and learning rate fading
Result of accepting new subject(s) from JAFFE database into
the system
Performance of the model tested for VeCAD-UTM fingervein database
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for VeCADUTM database
Parameter Setup for Case A and Case B
Accuracy for different combinations of weight initialization
algorithm and normalization methods for Case A VeCADUTM finger-vein database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for Case B VeCADUTM finger-vein database
Accuracy of each input size for VeCAD-UTM database
Accuracy achieved by different number of subjects
Benchmarking of accuracy achieved by the proposed method
and other CI methods for finger-vein identification system
Benchmarking of accuracy achieved by the proposed method
and conventional methods for finger-vein identification
system
Benchmarking on recognition speed for finger-vein biometric
system
110
111
112
120
120
120
121
121
122
123
124
125
125
126
xv
LIST OF FIGURES
FIGURE NO.
1.1
1.2
1.3
1.4
1.5
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
TITLE
Typical pattern recognition flow
Example of CNN Architecture
Taxonomy of biometric methods
Challenges faced by biometric face recognition system
Inappropriate lighting in finger-vein samples of VeCADUTM finger-vein database
Handwriting zip code application using CNN [4]
LeNet-5 CNN architecture for handwritting recognition
CNN architecture by Y. Simard et al. [5] with fused
convolution/subsampling
CNN architecture for fingerprint verification process [6]
Example of genuine and impostor pairs
Illustration of SCNN framework
A sample of ZIP code image and its possible segmentation
INSEG and OUTSEG segmentation for word recognition
Three schemes of SICoNNets. (a) Full-Connection. (b)
Toeplitz-Connection. (c) Binary-connection [7]
Shunting neuron model [7]
Sketch of the SPCNN connectivity [8].
Multiscale Convolutional Neural Networks (MSCNN) architecture for traffic sign recognition [9].
Three types of network paths connecting the input and the
hidden layers in the Multiple Circular Path Convolutional
Neural Network (MCPCNN) (a) Full-connection path
(b) Self-correlation path (c) Neighbourhood-correlation
path [10].
PAGE
1
4
5
12
13
17
18
18
28
29
30
32
32
33
33
34
35
36
xvi
2.14
2.15
2.16
2.17
3.1
3.2
3.3
3.4
3.5
3.6
4.1
4.2
4.3
4.4
4.5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
Multi-column Deep Neural Networks (MCDNN) and its basic
buiding block (a) Deep Neural Network (DNN) architecture
(b) The dataset is preprocessed (P) before the training
starts; during training all original or pre-processed images
are randomly distorted before each epoch (D). (c) MCDNN
architecture: the input image is pre-processed by n different
preprocessors P0 − Pn−1 and final predictions are obtained by
averaging individual predictions of each DNN [11].
Comparison between Two-Dimensional (2D) and ThreeDimensional (3D) convolutions
3D CNN architecture for human action recognition
Recurrent Convolutional Neural Networks (RCNN) architecture for dialog act classification
(a) Biological neuron (b) Modeling in NN
(a) Training phase (b) Test phase
An example of a LeNet-5 CNN architecture for handwritting
recognition [3]
Process in convolutional layer of CNN
Process in subsampling layer of CNN
LeNet-5 CNN architecture
Prediction of early stopping rule through cross-validation
technique [12].
CNN architecture by Y. Simard et al. [5] with fused
convolution/subsampling
The proposed four-layers CNN architecture
(a) convolution and subsampling of LeNet-5 CNN architecture and (b) fused convolution/subsampling in proposed
CNN
Conceptual view of generalizing the feature extraction layer
to other database in the system
Four different types of weight initialization methods
Sample of images from AT&T database
Sample of images from AR Purdue database
Preprocessing stages for AR Purdue database
Sample of images from FERET database
Preprocessing stages for FERET database
Validation error rates in cross-validation of the six tested
CNN models for AT&T
37
38
38
39
41
48
51
52
52
54
59
64
64
66
76
81
83
83
84
85
85
86
xvii
5.8
5.9
5.10
5.11
5.12
5.13
5.14
5.15
5.16
5.17
5.18
5.19
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
6.11
6.12
6.13
The proposed architecture for AT&T database (referred here
as the 5-14-60 model)
Feature maps at each layer after training for AT&T database
Validation error rates in cross-validation of the seven tested
CNN models for AR database
Misclassified samples for AR database
The proposed architecture for AR Purdue database (referred
here as the 15-45-130 model)
Feature maps at each layer after training for AR database
Validation error rates in cross-validation of the six tested
CNN models for FERET
The proposed architecture for FERET database (referred here
as the 15-47-130 model)
Feature maps at each layer after training for FERET database
Error rate on the training dataset
Error rate on the test dataset
Classification accuracy performance. (Note: Case A is for
the proposed learning with repeating scheme and Case B is
the original SDLM)
Non-ideal finger-vein samples of SDUMLA-HMT fingervein database
Sample of images from VeCAD-UTM database
Example of vertical cut
Example of horizontal cut
Image samples for 5 subjects (a) Case A and (b) Case B
Preprocessing stages and segmentation process for Case A
Preprocessing stages for Case B
Validation error rates in cross-validation of the six tested
CNN models for VeCAD-UTM finger-vein database
Training error for Case A and Case B
Test error for Case A and Case B
Rate of accuracy during training for Case A and Case B
The proposed model for VeCAD-UTM database (referred
here as the 5-13-50 model)
Output images at each layer after training for Case B
89
90
97
99
100
101
102
106
106
108
108
109
113
114
115
116
117
117
117
119
122
122
122
123
124
xviii
LIST OF ABBREVIATIONS
AI
–
Artificial Intelligence
ANN
–
Artificial Neural Network
BP
–
Backpropagation
CI
–
Computational Intelligence
CNNs
–
Convolutional Neural Networks
CMC
–
Cumulative Match Characteristic
DNN
–
Deep Neural Network
EBGM
–
Elastic Bunch Graph Matching
EER
–
Equal Error Rate
FERET
–
Face Recognition Technology
FAR
–
False Acceptance Rate
FRR
–
False Rejection Rate
FPGA
–
Field Programmable Gate Array
GA
–
Genetic Algorithm
GPU
–
Graphic Processing Unit
HD
–
Hausdorff Distance
HMM
–
Hidden Markov Model
LED
–
Light-Emitting Diode
LDA
–
Linear Discriminant Analysis
MATLAB
–
Matrix Laboratory
MSE
–
Mean Square Error
MHD
–
Modified Hausdorff Distance
MCDNN
–
Multi-column Deep Neural Networks
MLP
–
Multilayer Perceptron
MCPCNN
–
Multiple Circular Path Convolutional Neural Network
MSCNN
–
Multiscale Convolutional Neural Networks
NIR
–
Near Infrared
NNs
–
Neural Networks
xix
ORL
–
Olivetti Research Laboratory
1D
–
One-Dimensional
OS
–
Operating System
PC
–
Personal Computer
PIN
–
Personal Identification Number
PCA
–
Principal Component Analysis
PNN
–
Probabilistic Neural Network
RAM
–
Random Access Memory
ROC
–
Receiver Operating Characteristic
RCNN
–
Recurrent Convolutional Neural Networks
RNN
–
Recurrent Neural Networks
ROI
–
Region of Interest
RBF
–
Radial Basis Function
RF
–
Receptive Field
RPROP
–
Resilient Backpropagation
SOM
–
Self-Organizing Map
SICoNNets
–
Shunting Inhibitory Convolutional Neural Networks
SCNN
–
Siamese Convolutional Neural Network
SDNN
–
Space Displacement Neural Network
SPCNN
–
Sparse Convolutional Neural Network
SDLM
–
Stochastic Diagonal Levenberg-Marquardt
SVM
–
Support Vector Machine
3D
–
Three-Dimensional
TDNN
–
Time Delay Neural Networks
2D
–
Two-Dimensional
US
–
United State
UTM
–
Universiti Teknologi Malaysia
CHAPTER 1
INTRODUCTION
1.1
Overview of Pattern Recognition
Pattern recognition continues to be an active area of research since half a
century ago. The basic approach in pattern recognition is to transform raw images
through a series of image processing algorithms before applying the final stage of
classification. Examples of applications for pattern recognition includes: speech
recognition, handwriting recognition, object recognition, etc. Figure 1.1 shows a
common pattern recognition flow. The choice of sensors, preprocessing techniques
and decision making techniques depend on the characteristics of the problem domain.
Input
sample
Data
acquisition
Preprocessing
Segmentation
Dimensionality reduction
& Feature extraction
Classification
Result
Figure 1.1: Typical pattern recognition flow
The first stage of a pattern recognition system is data acquisition. In this
stage, raw data or images are collected from sensors or capture devices. Image
preprocessing is then performed, where transformations such as image enhancement,
image restoration, compression and morphological processing are applied [13]. Image
enhancement is applied to highlight certain features of interest in an image by
applying contrast transformation, Region of Interest (ROI) processing and noise
filtering. Then the image is restored by improving its appearance. This involves
recovering the original image that has been degraded by using a priori knowledge
of the degradation phenomenon and applying the inverse process. Image restoration
applies deblurring and noise reduction algorithm. Compression is then performed
to remove redundant data in the image. Typically, the image is converted to image
file formats such as JPEG (Joint Photographic Experts Group) image compression
standard. In morphological processing stage, it involves with the techniques to extract
FINGER-VEIN BIOMETRIC IDENTIFICATION
SYAFEEZA AHMAD RADZI
UNIVERSITI TEKNOLOGI MALAYSIA
© Universiti Teknikal Malaysia Melaka
PSZ 19:16 (Pind. 1/07)
UNIVERSITI TEKNOLOGI MALAYSIA
DECLARATION OF THESIS / UNDERGRADUATE PROJECT PAPER AND COPYRIGHT
Author’s full name :
Date of birth
:
Title
:
SYAFEEZA BINTI AHMAD RADZI
24TH MARCH 1981
CONVOLUTIONAL NEURAL NETWORKS FOR FACE RECOGNITION
AND FINGER-VEIN BIOMETRIC IDENTIFICATION
Academic Session:
2013/2014/2
I declare that this thesis is classified as :
√
CONFIDENTIAL
(Contains confidential information under the Official Secret
Act 1972)*
RESTRICTED
(Contains restricted information as specified by the
organisation where research was done)*
OPEN ACCESS
I agree that my thesis to be published as online open access
(full text)
I acknowledged that Universiti Teknologi Malaysia reserves the right as follows :
1. The thesis is the property of Universiti Teknologi Malaysia.
2. The Library of Universiti Teknologi Malaysia has the right to make copies for the purpose
of research only.
3. The Library has the right to make copies of the thesis for academic exchange.
Certified by :
SIGNATURE
810324-71-5024
SIGNATURE OF SUPERVISOR
MOHAMED KHALIL BIN MOHD HANI
(NEW IC NO. /PASSPORT NO.)
NAME OF SUPERVISOR
Date : 2 JULY 2014
NOTES :
*
Date : 2 JULY 2014
If the thesis is CONFIDENTIAL or RESTRICTED, please attach with the letter from
the organisation with period and reasons for confidentiality or restriction.
“I hereby declare that I have read this thesis and in my
opinion this thesis is sufficient in terms of scope and quality for the
award of the degree of Doctor of Philosophy (Electrical Engineering)”
Signature
:
Name
:
Prof. Dr. Mohamed Khalil Mohd. Hani
Date
:
July 2, 2014
BAHAGIAN A - Pengesahan Kerjasama*
Adalah disahkan bahawa projek penyelidikan tesis ini telah dilaksanakan melalui kerjasama
antara _ _ _ _ _ _ _ _ _ _ dengan _ _ _ _ _ __ _ __
Disahkan oleh:
Tandatangan
Tarikh:
Nama
Jawatan
(Cop rasmi)
* Jika penyediaan tesislprojek melibatkan kerjasama.
BAHAGIAN B- Untuk Kegunaan Pejabat Sekolah Pengajian Siswazah
Tesis ini telah diperiksa dan diakui oleh:
Nama dan Alamat Pemeriksa Luar
Prof. Dr. Mohd Alauddin Mohd Ali
Department of Electrical Electronic & Systems
Engineering,
Faculty of Engineering and Built Environment,
Universiti Kebangsaan Malaysia,
43600 Bangi, Se!ang-0>r.
Nama dan Alamat Pemeriksa Dalam
Prof. Dr. Jasmy bin Yunus
Dekan
Fakulti Biosains dan Kejuruteraan Perubatan,
UTM Johor Bahru.
Disahkan oleh Timbalan Pendaftar di Sekolah Pengajian Siswazah:
Tandatangan :
Nama
Tarikh :
ASRAM BIN SULAIMAN @ SAIM
© Universiti Teknikal Malaysia Melaka
CONVOLUTIONAL NEURAL NETWORKS FOR FACE RECOGNITION AND
FINGER-VEIN BIOMETRIC IDENTIFICATION
SYAFEEZA AHMAD RADZI
A thesis submitted in fulfilment of the
requirements for the award of the degree of
Doctor of Philosophy (Electrical Engineering)
Faculty of Electrical Engineering
Universiti Teknologi Malaysia
JULY 2014
© Universiti Teknikal Malaysia Melaka
ii
I declare that this thesis entitled "Convolutional Neural Networks for Face Recognition
and Finger-vein Biometric Identification" is the result of my own research except as
cited in the references. The thesis has not been accepted for any degree and is not
concurrently submitted in candidature of any other degree.
Signature
Name
SYAFEEZA AHMAD RADZI
Date
July 2, 2014
© Universiti Teknikal Malaysia Melaka
iii
Dedicated to my beloved parents, husband and daughter.
iv
ACKNOWLEDGEMENT
I am very blessed with the completion of this thesis. With this opportunity,
I would like to express my gratitude to several people that have involved directly or
indirectly to the completion of this thesis.
Firstly, I would like to express my sincere appreciation to my supervisor
Professor Dr. Mohamed Khalil bin Mohd Hani for the useful critics, guidance and
advices during the journey. I have gained valuable knowledge during his supervision.
It was also a privilege to work closely with the members of VeCAD Lab namely Rabia,
Vishnu, Lee Yee Hui, Sia Chen Wei, Moganesh and Liew Shan Sung. I am very
fortunate to have these people during my ups and downs of conducting this research. I
am glad for the sharing of constructive ideas and their concern.
Most importantly, I would like to thank my family especially my parents,
husband and daughter. Thank you for the continuous encouragement, patience and
boundless support in all my endeavours.
Last but not least, I would like to acknowledge Universiti Teknikal Malaysia
Melaka (UTeM) for giving me the opportunity to pursue my study up to this level.
v
ABSTRACT
The Convolutional Neural Network (CNN), a variant of the Multilayer
Perceptron (MLP), has shown promise in solving complex recognition problems,
particularly in visual pattern recognition. However, the classical LeNet-5 CNN model,
which most solutions are based on, is highly compute-intensive. This CNN also suffers
from long training time, due to the large number of layers that ranges from six to
eight. In this research, a CNN model with a reduced complexity is proposed for
application in face recognition and finger-vein biometric identification. A simpler
architecture is obtained by fusing convolutional and subsampling layers into one
layer, in conjunction with a partial connection scheme applied between the first two
layers in the network. As a result, the total number of layers is reduced to four.
The number of feature maps at each layer is optimized according to the type of
image database being processed. Consequently, the numbers of network parameters
(including neurons, trainable parameters and connections) are significantly reduced,
essentially increasing the generalization ability of the network. The Stochastic
Diagonal Levenberg-Marquadt (SDLM) backpropagation algorithm is modified and
applied in the training of the proposed network. With this learning algorithm, the
convergence rate is accelerated such that the proposed CNN converges within 15
epochs. For face recognition, the proposed CNN achieves recognition rates of 100.00%
and 99.50% for AT&T and AR Purdue face databases respectively. Recognition time
on the AT&T database is less than 0.003 seconds. These results outperform previous
existing works. In addition, when compared with the other CNN-based face recognizer,
the proposed CNN model has the least number of network parameters, hence better
generalization ability. A training scheme is also proposed to recognize new categories
without full CNN training. In this research, a novel CNN solution for the finger-vein
biometric identification problem is also proposed. To the best of knowledge, there is
no previous work reported in literature that applied CNN for finger-vein recognition.
The proposed method is efficient in that simple preprocessing algorithms are deployed.
The CNN design is adapted on a finger-vein database, which is developed in-house and
contains 81 subjects. A recognition accuracy of 99.38% is achieved, which is similar
to the results of state-of-the-art work. In conclusion, the success of the research in
solving face recognition and finger-vein biometric identification problems proves the
feasibility of the proposed CNN model in any pattern recognition system.
vi
ABSTRAK
Convolutional Neural Network (CNN) yang merupakan variasi kepada
Multilayer Perceptron (MLP) telah menunjukkan kebolehan dalam kerja pengecaman
yang rumit terutamanya dalam pengecaman corak visual. Walau bagaimanapun,
senibina klasik CNN iaitu LeNet-5, yang merupakan asas kepada kebanyakan
penyelesaian, mempunyai pengiraan intensif yang tinggi. CNN ini juga berhadapan
dengan masa latihan yang terlalu lama disebabkan oleh bilangan lapisannya dalam
lingkungan enam hingga lapan lapisan. Dalam kajian ini, model CNN dengan
kurang kekompleksan telah dicadangkan untuk diaplikasi pada pengecaman muka dan
pengesahan identiti biometrik urat jari. Senibina yang lebih ringkas telah diperolehi
dengan cara mencantumkan lapisan convolution dan subsampling ke satu lapisan,
dengan gabungan skim sambungan separa antara dua lapisan pertama dalam rangkaian.
Keputusannya, jumlah lapisan telah dikurangkan kepada empat. Bilangan petak sifat
pada setiap lapisan telah dioptimumkan berdasarkan kepada jenis pangkalan data yang
digunakan. Kesannya, bilangan parameter rangkaian (termasuk neuron, parameter
terlatih dan sambungan) nyata sekali dapat dikurangkan, terutamanya kebolehan
generalisasi yang lebih baik. Algoritma Stochastic Diagonal Levenberg-Marquadt
(SDLM) telah diubah suai dan diaplikasi dalam latihan rangkaian yang dicadangkan.
Dengan algoritma ini, kadar pembelajaran titik tumpu telah dipercepatkan untuk tumpu
dalam tempoh 15 epoch. Untuk pengecaman muka, CNN yang dicadangkan mencapai
kadar pengecaman sebanyak 100.00% dan 99.50% masing-masing untuk pangkalan
data AT&T dan AR Purdue. Masa pengecaman untuk AT&T adalah kurang daripada
0.003 saat. Keputusan yang diperolehi telah mengatasi kerja terdahulu. Tambahan
pula, apabila dibandingkan dengan reka bentuk CNN yang lain, senibina CNN yang
diusulkan mempunyai parameter rangkaian yang paling sedikit malahan ia mempunyai
kebolehan generalisasi yang lebih baik. Satu skim latihan juga telah dicadangkan untuk
mengecam kategori baru tanpa memerlukan keseluruhan latihan CNN. Dalam kajian
ini, penyelesaian CNN untuk masalah pengesahan identiti biometrik urat jari juga
telah dicadangkan. Sepanjang pengetahuan yang ada, tiada kerja sebelumnya yang
dilaporkan mengaplikasi CNN untuk pengesahan identiti biometrik urat jari. Kaedah
yang dicadangkan berkesan kerana algoritma pemprosesan mudah digunakan. Reka
bentuk CNN diadaptasi pada pangkalan data urat jari, yang telah dihasilkan sendiri dan
mengandungi 81 orang. Kejituan pengecaman sebanyak 99.38% telah dicapai, yang
hampir sama dengan keputusan yang diperoleh daripada kerja terkini. Kesimpulannya,
kajian ini telah berjaya menyelesaikan masalah pengecaman muka dan pengesahan
identiti biometrik urat jari membuktikan bahawa CNN yang dicadangkan boleh
dilaksanakan dalam sebarang sistem pengecaman corak.
vii
TABLE OF CONTENTS
CHAPTER
TITLE
DECLARATION
DEDICATION
ACKNOWLEDGEMENT
ABSTRACT
ABSTRAK
TABLE OF CONTENTS
LIST OF TABLES
LIST OF FIGURES
LIST OF ABBREVIATIONS
1
2
INTRODUCTION
1.1
Overview of Pattern Recognition
1.2
Neural Networks in Pattern Recognition Problems
1.3
Convolutional Neural Network
1.4
Biometric Pattern Recognition
1.4.1
Summary of Existing Face Recognizers
1.4.2
Summary of Work on Finger-vein Pattern
Recognition
1.5
Problem Statement
1.6
Research Objectives
1.7
Scope of Work
1.8
Research Contributions
1.9
Thesis Organization
LITERATURE REVIEW
2.1
Basic CNN Architectures
2.2
Previous Work applying CNN in Face Recognition
2.3
Related Previous Work on Finger-vein Biometric
Identification
PAGE
ii
iii
iv
v
vi
vii
xii
xv
xviii
1
1
2
3
4
7
8
8
13
14
14
16
17
17
19
23
viii
2.4
2.5
3
4
Other State-of-the-Art CNN Architectures
2.4.1
Siamese CNN
2.4.2
Space Displacement Neural Networks
2.4.3
Shunting Inhibitory Convolutional Neural
Networks
2.4.4
Sparse Convolutional Neural Networks
2.4.5
Multiscale Convolutional Neural Networks
2.4.6
Multiple Circular Path Convolutional
Neural Networks
2.4.7
Multi-column Deep Neural Networks
2.4.8
3D Convolutional Neural Networks
2.4.9
Recurrent Convolutional Neural Networks
Summary
CONVOLUTIONAL NEURAL NETWORK - ALGORITHMS AND ARCHITECTURE
3.1
Multilayer Perceptron Neural Network (MLPNN)
3.1.1
Activation Functions
3.1.2
Weight Initialization Methods
3.1.3
Normalization Method
3.2
Neural Network Training
3.2.1
Gradient Descent Backpropagation Algorithm
3.2.2
Resilient Backpropagation
3.3
LeNet-5 Convolutional Neural Networks
3.3.1
Algorithms in LeNet-5
3.3.2
Convolutional Neural Network Training
3.4
Summary
PROPOSED
CONVOLUTIONAL
NEURAL
NETWORK: MODELLING, ARCHITECTURE AND
ALGORITHM
4.1
Methodology
4.1.1
Cross-validation Technique to optimize
Generalization Ability
4.1.2
Techniques to Improve Convergence Rate
27
27
31
32
34
34
35
36
37
39
39
40
40
42
42
44
45
46
49
49
50
54
55
57
57
57
59
ix
4.2
4.3
4.4
4.5
4.6
5
4.1.3
Winner-Takes-All Rule
Proposed CNN Model
4.2.1
Introduction
4.2.2
CNN with Fused Convolution/Subsampling
4.2.3
Connection Schemes
Proposed Training Algorithm
4.3.1
Stochastic
Diagonal
LevenbergMarquardt (SDLM)
4.3.2
Proposed Modifications to SDLM Algorithm
Proposed Training Scheme to recognize New
Categories without Full CNN Training
Generalizing the CNN Architecture for New
Problem Domain
Summary
EXPERIMENTAL WORK AND RESULTS FOR CNN IN
FACE RECOGNITION
5.1
Performance Evaluation Procedures
5.2
Data Preparation
5.2.1
Face databases
5.3
Experimental Work and Results of the Proposed
Baseline CNN Model
5.3.1
Results of Cross-validation
5.3.2
Best Connection Scheme
5.3.3
The Most Suitable Normalization and
Weight Initialization Methods
5.3.4
The Optimum Input Image Size
5.3.5
Benchmarking Results
5.3.5.1 Benchmarking of Accuracy
5.3.5.2 Benchmarking of Execution
and Convergence Time
5.3.5.3 Analysis and Benchmarking of
Model Complexity
5.3.5.4 Comparison between the proposed CNN Model and LeNet-5
5.4
Experimental Work and Results of Proposed CNN
Model
62
63
63
63
67
69
69
71
73
76
77
78
78
82
82
85
86
87
88
89
90
90
92
93
95
96
x
5.4.1
5.4.2
5.4.3
5.5
5.6
5.7
5.8
6
Results of Cross-validation
Best Connection Scheme
The Most Suitable Normalization and
Weight Initialization Methods
5.4.4
The Optimum Input Image Size
5.4.5
Benchmarking Results
Experimental Results of CNN Model Targeted for
Face Images of the ”Extreme Challenge” Type
5.5.1
Results of Cross-validation
5.5.2
Best Connection Scheme
5.5.3
The Most Suitable Normalization and
Weight Initialization Methods
5.5.4
The Optimum Input Image Size
5.5.5
Benchmarking Results
Analysis of the Proposed Modification on SDLM
Algorithm
5.6.1
Comparison between the Proposed Modification on SDLM Algorithm and Standard Backpropagation
Analysis of Training Scheme to recognize New
Categories without Full CNN Training
Conclusion
EXPERIMENTAL WORK AND RESULTS FOR CNN IN
FINGER-VEIN BIOMETRIC IDENTIFICATION
6.1
Generation of Finger-vein Database for Biometric
Identification
6.2
Performance Evaluation of Finger-vein Biometric
Identification
6.2.1
VeCAD-UTM database
6.2.2
Results of Cross-validation
6.2.3
Best Connection Scheme
6.2.4
The Most Suitable Normalization and
Weight Initialization Methods
6.2.5
The Optimum Input Image Size
6.2.6
Benchmarking Results
6.3
Conclusion
96
96
97
99
100
101
101
103
104
105
107
107
109
111
112
113
113
118
118
118
120
120
123
124
126
xi
7
CONCLUSION
7.1
Concluding Remarks
7.2
Suggestions for Future Work
127
127
130
REFERENCES
132
Appendix A
146
xii
LIST OF TABLES
TABLE NO.
1.1
1.2
1.3
1.4
1.5
1.6
2.1
2.2
2.3
2.4
2.5
3.1
3.2
4.1
4.2
4.3
4.4
4.5
5.1
5.2
5.3
TITLE
Characteristic comparison of biometric application [1]
History of some biometric development [2]
Previous work on face recognition based on CNN
Previous work on face recognition applying non-CNN
methods
Accuracy achieved by Computational Intelligence (CI)
approach for finger-vein identification system
Detail information and accuracy achieved by conventional
approach for fingervein identification system
Face Recognition System using CNN
Conventional methods of finger-vein identification
Algorithms applied by CI approach for finger-vein identification system
Accuracy achieved by CI-based for finger-vein identification
Related previous work on finger-vein biometric system and
its recognition speed
Input/output relation and symbol for each activation function
Each row indicates which feature maps in S2 are combined
by the particular feature maps of C3 [3]
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for AT&T
database
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for AR Purdue
database
The values of ηg as given in [3]
Pose variations of FERET database
Performance of the model tested for AT&T database
Variant type of connections between C1 and C2 layer
PAGE
6
6
7
7
8
8
22
25
26
27
27
43
53
68
68
69
69
73
85
87
87
xiii
5.4
5.5
5.6
5.7
5.8
5.9
5.10
5.11
5.12
5.13
5.14
5.15
5.16
5.17
5.18
5.19
5.20
5.21
5.22
5.23
5.24
5.25
5.26
5.27
5.28
5.29
Connection scheme between C1 and C2 layer for AT&T
database
Parameter Setup for AT&T database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for AT&T database
Accuracy of each input size for AT&T database
Benchmarking of accuracy of other CNN face recognizers on
AT&T database
Benchmarking with non-CNN approach
Benchmarking of execution time and convergence learning
time against other face recognizers
Benchmarking of model complexity against previous work
that apply CNN (not restricted to face recognition problems)
Comparison between fused architecture and LeNet-5
Performance of the model tested for AR database
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for AR Purdue
database
Parameter Setup for AR Purdue database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for AR Purdue database
Parameter setup for each input size for AR Purdue database
Accuracy of each input size for AR Purdue database
Benchmarking of face identification system using AR Purdue
database with existing works
Performance of the model tested for FERET database
Variant type of connections between C1 and C2 layer for
FERET database
Connection scheme between C1 and C2 layer for FERET
database
Parameter Setup for FERET database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for FERET database
Parameter setup for each input size for FERET database
Accuracy of each input size for FERET database
Benchmarking of face identification system using FERET
database with existing works
Experimental setup for standard Backpropagation (BP)
88
88
88
89
91
91
92
95
96
97
98
98
99
99
100
100
102
103
103
104
104
105
105
105
107
110
xiv
5.30
5.31
5.32
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
6.11
Result of BP training using combination of different initial
learning rate and learning rate fading
Result of BP training using combination of different initial
learning rate and learning rate fading
Result of accepting new subject(s) from JAFFE database into
the system
Performance of the model tested for VeCAD-UTM fingervein database
Variant type of connections between C1 and C2 layer
Connection scheme between C1 and C2 layer for VeCADUTM database
Parameter Setup for Case A and Case B
Accuracy for different combinations of weight initialization
algorithm and normalization methods for Case A VeCADUTM finger-vein database
Accuracy for different combinations of weight initialization
algorithm and normalization methods for Case B VeCADUTM finger-vein database
Accuracy of each input size for VeCAD-UTM database
Accuracy achieved by different number of subjects
Benchmarking of accuracy achieved by the proposed method
and other CI methods for finger-vein identification system
Benchmarking of accuracy achieved by the proposed method
and conventional methods for finger-vein identification
system
Benchmarking on recognition speed for finger-vein biometric
system
110
111
112
120
120
120
121
121
122
123
124
125
125
126
xv
LIST OF FIGURES
FIGURE NO.
1.1
1.2
1.3
1.4
1.5
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12
2.13
TITLE
Typical pattern recognition flow
Example of CNN Architecture
Taxonomy of biometric methods
Challenges faced by biometric face recognition system
Inappropriate lighting in finger-vein samples of VeCADUTM finger-vein database
Handwriting zip code application using CNN [4]
LeNet-5 CNN architecture for handwritting recognition
CNN architecture by Y. Simard et al. [5] with fused
convolution/subsampling
CNN architecture for fingerprint verification process [6]
Example of genuine and impostor pairs
Illustration of SCNN framework
A sample of ZIP code image and its possible segmentation
INSEG and OUTSEG segmentation for word recognition
Three schemes of SICoNNets. (a) Full-Connection. (b)
Toeplitz-Connection. (c) Binary-connection [7]
Shunting neuron model [7]
Sketch of the SPCNN connectivity [8].
Multiscale Convolutional Neural Networks (MSCNN) architecture for traffic sign recognition [9].
Three types of network paths connecting the input and the
hidden layers in the Multiple Circular Path Convolutional
Neural Network (MCPCNN) (a) Full-connection path
(b) Self-correlation path (c) Neighbourhood-correlation
path [10].
PAGE
1
4
5
12
13
17
18
18
28
29
30
32
32
33
33
34
35
36
xvi
2.14
2.15
2.16
2.17
3.1
3.2
3.3
3.4
3.5
3.6
4.1
4.2
4.3
4.4
4.5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
Multi-column Deep Neural Networks (MCDNN) and its basic
buiding block (a) Deep Neural Network (DNN) architecture
(b) The dataset is preprocessed (P) before the training
starts; during training all original or pre-processed images
are randomly distorted before each epoch (D). (c) MCDNN
architecture: the input image is pre-processed by n different
preprocessors P0 − Pn−1 and final predictions are obtained by
averaging individual predictions of each DNN [11].
Comparison between Two-Dimensional (2D) and ThreeDimensional (3D) convolutions
3D CNN architecture for human action recognition
Recurrent Convolutional Neural Networks (RCNN) architecture for dialog act classification
(a) Biological neuron (b) Modeling in NN
(a) Training phase (b) Test phase
An example of a LeNet-5 CNN architecture for handwritting
recognition [3]
Process in convolutional layer of CNN
Process in subsampling layer of CNN
LeNet-5 CNN architecture
Prediction of early stopping rule through cross-validation
technique [12].
CNN architecture by Y. Simard et al. [5] with fused
convolution/subsampling
The proposed four-layers CNN architecture
(a) convolution and subsampling of LeNet-5 CNN architecture and (b) fused convolution/subsampling in proposed
CNN
Conceptual view of generalizing the feature extraction layer
to other database in the system
Four different types of weight initialization methods
Sample of images from AT&T database
Sample of images from AR Purdue database
Preprocessing stages for AR Purdue database
Sample of images from FERET database
Preprocessing stages for FERET database
Validation error rates in cross-validation of the six tested
CNN models for AT&T
37
38
38
39
41
48
51
52
52
54
59
64
64
66
76
81
83
83
84
85
85
86
xvii
5.8
5.9
5.10
5.11
5.12
5.13
5.14
5.15
5.16
5.17
5.18
5.19
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
6.11
6.12
6.13
The proposed architecture for AT&T database (referred here
as the 5-14-60 model)
Feature maps at each layer after training for AT&T database
Validation error rates in cross-validation of the seven tested
CNN models for AR database
Misclassified samples for AR database
The proposed architecture for AR Purdue database (referred
here as the 15-45-130 model)
Feature maps at each layer after training for AR database
Validation error rates in cross-validation of the six tested
CNN models for FERET
The proposed architecture for FERET database (referred here
as the 15-47-130 model)
Feature maps at each layer after training for FERET database
Error rate on the training dataset
Error rate on the test dataset
Classification accuracy performance. (Note: Case A is for
the proposed learning with repeating scheme and Case B is
the original SDLM)
Non-ideal finger-vein samples of SDUMLA-HMT fingervein database
Sample of images from VeCAD-UTM database
Example of vertical cut
Example of horizontal cut
Image samples for 5 subjects (a) Case A and (b) Case B
Preprocessing stages and segmentation process for Case A
Preprocessing stages for Case B
Validation error rates in cross-validation of the six tested
CNN models for VeCAD-UTM finger-vein database
Training error for Case A and Case B
Test error for Case A and Case B
Rate of accuracy during training for Case A and Case B
The proposed model for VeCAD-UTM database (referred
here as the 5-13-50 model)
Output images at each layer after training for Case B
89
90
97
99
100
101
102
106
106
108
108
109
113
114
115
116
117
117
117
119
122
122
122
123
124
xviii
LIST OF ABBREVIATIONS
AI
–
Artificial Intelligence
ANN
–
Artificial Neural Network
BP
–
Backpropagation
CI
–
Computational Intelligence
CNNs
–
Convolutional Neural Networks
CMC
–
Cumulative Match Characteristic
DNN
–
Deep Neural Network
EBGM
–
Elastic Bunch Graph Matching
EER
–
Equal Error Rate
FERET
–
Face Recognition Technology
FAR
–
False Acceptance Rate
FRR
–
False Rejection Rate
FPGA
–
Field Programmable Gate Array
GA
–
Genetic Algorithm
GPU
–
Graphic Processing Unit
HD
–
Hausdorff Distance
HMM
–
Hidden Markov Model
LED
–
Light-Emitting Diode
LDA
–
Linear Discriminant Analysis
MATLAB
–
Matrix Laboratory
MSE
–
Mean Square Error
MHD
–
Modified Hausdorff Distance
MCDNN
–
Multi-column Deep Neural Networks
MLP
–
Multilayer Perceptron
MCPCNN
–
Multiple Circular Path Convolutional Neural Network
MSCNN
–
Multiscale Convolutional Neural Networks
NIR
–
Near Infrared
NNs
–
Neural Networks
xix
ORL
–
Olivetti Research Laboratory
1D
–
One-Dimensional
OS
–
Operating System
PC
–
Personal Computer
PIN
–
Personal Identification Number
PCA
–
Principal Component Analysis
PNN
–
Probabilistic Neural Network
RAM
–
Random Access Memory
ROC
–
Receiver Operating Characteristic
RCNN
–
Recurrent Convolutional Neural Networks
RNN
–
Recurrent Neural Networks
ROI
–
Region of Interest
RBF
–
Radial Basis Function
RF
–
Receptive Field
RPROP
–
Resilient Backpropagation
SOM
–
Self-Organizing Map
SICoNNets
–
Shunting Inhibitory Convolutional Neural Networks
SCNN
–
Siamese Convolutional Neural Network
SDNN
–
Space Displacement Neural Network
SPCNN
–
Sparse Convolutional Neural Network
SDLM
–
Stochastic Diagonal Levenberg-Marquardt
SVM
–
Support Vector Machine
3D
–
Three-Dimensional
TDNN
–
Time Delay Neural Networks
2D
–
Two-Dimensional
US
–
United State
UTM
–
Universiti Teknologi Malaysia
CHAPTER 1
INTRODUCTION
1.1
Overview of Pattern Recognition
Pattern recognition continues to be an active area of research since half a
century ago. The basic approach in pattern recognition is to transform raw images
through a series of image processing algorithms before applying the final stage of
classification. Examples of applications for pattern recognition includes: speech
recognition, handwriting recognition, object recognition, etc. Figure 1.1 shows a
common pattern recognition flow. The choice of sensors, preprocessing techniques
and decision making techniques depend on the characteristics of the problem domain.
Input
sample
Data
acquisition
Preprocessing
Segmentation
Dimensionality reduction
& Feature extraction
Classification
Result
Figure 1.1: Typical pattern recognition flow
The first stage of a pattern recognition system is data acquisition. In this
stage, raw data or images are collected from sensors or capture devices. Image
preprocessing is then performed, where transformations such as image enhancement,
image restoration, compression and morphological processing are applied [13]. Image
enhancement is applied to highlight certain features of interest in an image by
applying contrast transformation, Region of Interest (ROI) processing and noise
filtering. Then the image is restored by improving its appearance. This involves
recovering the original image that has been degraded by using a priori knowledge
of the degradation phenomenon and applying the inverse process. Image restoration
applies deblurring and noise reduction algorithm. Compression is then performed
to remove redundant data in the image. Typically, the image is converted to image
file formats such as JPEG (Joint Photographic Experts Group) image compression
standard. In morphological processing stage, it involves with the techniques to extract