License Plate Recognition System Using Video And Images.

(1)

LICENSE PLATE RECOGNITION SYSTEM USING VIDEO AND

IMAGE

ZURAINEY BINTI CHE ARSAD

(2)

“I hereby declared that I have read through this report and found that it has comply the partial fulfillment for awarding the degree of Bachelor of Electrical Engineering

(Control, Instrumentation & Automation)”.

Signature : ……….

Supervisor‟s Name : Mr. MAZREE BIN IBRAHIM

(3)

LICENSE PLATE RECOGNITION SYSTEM USING VIDEO AND IMAGES

ZURAINEY BINTI CHE ARSAD

This Report Is Submitted In Partial Fulfillment of Requirements for the Degree of Bachelor in Electrical Engineering (Control, Instrumentation & Automation)

Fakulti Kejuruteraan Elektrik Universiti Teknikal Malaysia Melaka

(4)

iii

“I hereby declared that this report is a result of my own work except for the excerpts that have been cited clearly in the references”.

Signature : ………

Name : ZURAINEY CHE ARSAD

(5)

ACKNOWLEGDMENT

First of all, I would like to take this opportunity to extend my deepest gratitude to all those who have assisted me in making this report successfully.

My sincere thanks to my respected and supportive supervisor, Mr. Mazree b. Ibrahim and Mrs. Hamimi Fadziati for their patience and guidance throughout the duration of this project.

I would also like to take this opportunity to thank all the UTeM lecturers for their support and guidance throughout the course of the project. Last but not least, I would like to thank all my friends and family who given me spirit and support during my project.

Saying thank you is surely inadequate, yet most appropriate at this juncture. To the above mentioned, this report seals the author‟s gratitude to you all.

(6)

ABSTRACT

The purpose of this project was to build a real time application which recognizes license plates from cars at gate, for example at entrance of a parking area. The system, based on regular PC with video camera, catches video frames which include a visible car license plate and processes them. Once a license plate is detected, its digits are recognized, displayed on the User Interface or checked against database. The focus is on the design of algorithms used for extracting the license plate from a single image, isolating the characters of the plate and identifying the individual character. Image sample of vehicle‟s license plate is acquisited by video camera, then, it is converted into Black and White (BW). Artificial Neural Network (ANN) is applied to recognize the character‟s plate. Learning method of the ANN is back propagation and all process is handled by Matlab version R2006a 7.1.

(7)

ABSTRAK

Projek ini dijalankan adalah bertujuan untuk mengaplikasikan kepada suasana ataupun kepada keadaan yang sebenar iaitu mengenai pengenalpastian nombor pendaftaran kenderaan, contohnya seperti pada pintu masuk di kawasan tempat letak kereta. Sistem ini menggunakan aplikasi komputer dan juga kamera video yang mana akan menangkap gambar, iaitu gambar nombor pendaftaran kenderaan dan akan memproseskannya. Apabila nombor pendaftaran kenderaan telah dikesan, digit-digit yang ada pada nombor pendaftaran kenderaan tersebut akan dikenalpasti, dan dipaparkan pada User Interface ataupun membuat pemeriksaan pada database. Focus projek ini adalah untuk mereka algoritme untuk “extracting” nombor pendaftaran kenderaan kepada imej yang lebih mudah untuk dikenalpasti, memisahkan semua ciri-ciri pada nombor pendaftaran dan mengenalpasti setiap abjad dan nombor. Nombor pendaftaran kenderaan ini akan ditangkap dengan kamera video, dan selepas itu, akan ditukarkan kepada warna hitam dan putih. Jaringan Neural digunakan untuk mengenalpasti semua cirri-ciri yang ada pada nombor pendaftaran kenderaan. Kaedah pembelajaran di dalam Jaringan Neural adalah “back propagation” dan semua proses akan menggunakan Matlab versi R2006a 7.1.

(8)

vii

CONTENTS

CHAPTER CONTENTS PAGE

SUPERVISOR RECOGNITION i

PROJECT TITLE ii

RECOGNITION iii

ACKNOWLEDGEMENT iv

ABSTRACT v-vi

CONTENTS vii

LIST OF TABLE xi

LIST OF FIGURE xii

LIST OF APPENDIX x

LIST OF ABBREVIATION xvi

I INTRODUCTION

1.0 Introduction 1

1.1 Objective 2

1.2 Problem Statement 2

1.3 Scope Project 3

II LITERATURE REVIEW

2.1 Purpose statement for Neural Network approach 4

2.2 Artificial Neural Network 4-5

2.3 Recognition 5-6

(9)

viii

III THEORY

3.1 Chapter Overview 8

3.2 Malaysian Vehicle License Plate 8 3.3 Basic design of license plate 9

3.4 Software R2006a 9-10

3.5 Definition of Image Processing 10

3.6 What is Neural Network 11-12

3.6.1 Multilayer Perceptron (MLP) 12-13 3.6.2 Neural Network element 13 3.6.3 Neural Network Architecture 14

IV METHODOLOGY

4.1 Chapter Overview 15

4.2 Development of program codes 15

4.3 Capturing image 16

4.3.1 GUI (Graphical User Interface) 17-18 for capturing image

4.3.2 Video Setup 18-19

4.3.3 Previewing video 20

4.3.4 Capturing images 20

4.4 Recognizing character on the car plate 20

4.5 Simulink 21

4.5.1 Part 1: Image acquisition and object 21-24 Recognition

4.5.2 Part 2: Design the progression of the system 24-25 4.5.3 Part 3: Description of the implementation 26

4.5.3.1 License plate extraction 26 4.5.3.2Black region extraction 26

(10)

4.5.3.3Dilating black regions 27 4.5.3.4Determining the angle of the plate 27 4.5.3.5Improved license plate region 28 4.5.3.6License Plate Crop 28 4.5.3.7License Plate Quantization and 29 Equalization/ converts RGB image to BW 4.5.4 Part 4: Developed Simulink.model 29-30

4.6 Neural Network 31

4.6.1 Part 1: Design character recognition of 31 alphabet and number

4.6.2 Part 2: Training and recognize the character 31-33 4.6.2.1 Initialization 33

4.6.2.2Training 33

4.6.2.3Training without noise 34 4.6.2.4Training with noise 34 4.6.3 Part 3: Training a NN for classification 35

4.6.3.1Read the image 35

4.6.3.2Image Preprocessing 35 4.6.3.3Creating Vectors data for the 35

Neural Network (objects)

4.6.3.4Creating and training of the 36 Neural Network

4.6.3.5Testing the Neural Network 36

4.7Using GUI (Graphical User Interface) 37-42

4.8Assemble 42

(11)

V RESULT

5.1 Capturing image using web camera 44-45

5.2 Result from RGB image 45-46

5.3 Neural Network 46

5.4 Noisy letter A to Z and the network 47-54 recognize the character

5.5 Error for network to recognize 55 5.6 Noisy letter 1 to 9 and the network 56-58

recognized the character

5.7 GUI 59-63

VI DISCUSSION AND SUGGESTION

6.1 Discussion and Conclusion 64

6.2 Suggestion 65

VII CONCLUSION

7.1 Conclusion 66

REFERENCES 67-68

APPENDIX A CAPTURING IMAGE 69

APPENDIX B NEURAL NETWORK 73

(12)

LIST OF TABLE

NO. TITLE PAGE

(13)

xii

LIST OF FIGURE

NO. TITLE PAGE

2.1 Some graphics of transfer function 5

2.2 Extraction of a character in a matrix 7 2.3 Identification of background and character. 7

3.1 A graphical representation of an MLP 12

3.2 Training process 13

3.3 Neural Network architecture 14

4.1 Block diagram show 2 methods in this project 15 4.2 Block diagram show sequence method for capturing image. 16

4.3 Command window 17

4.4 Guide Quick Start 17

4.5 Create figure using button 18

4.6 Create figure using 3 pushbutton and 1 axes 18

4.7 Command window for adaptor 19

4.8 Block diagram show the sequence method of recognizing 20 4.9 Block diagram show the sequence method of Simulink 21 4.10 Image Acquisition block-mask parameters 22 4.11 Color Space Conversion block-mask parameter 23

4.12 Autothreshold block parameter 24

4.13 Block diagram of the system 24

4.14 Block diagram of license plate extraction and character

Segmentation 25

4.15 Example of captured frame 26

4.16 Captured frame with black region dilated 27

(14)

xiii

4.18 Improved License Plate region 28

4.19 License Plate Crop 28

4.20 Gray scale input license plate 29

4.21 Overall Simulink Program 30

4.22 Overall Program for Neural Network 31

4.23 Alphabet A without noise 32

4.24 Alphabet A with noise 32

4.25 Block diagram of the sequence method using GUI 37 4.26 Design the drawing using axes, panel button and

pushbutton 38

4.27 Save and Run the M-file 39

4.28 Block diagram of the sequence method using

pushbutton load 39

4.29 Block diagram of the sequence method using

pushbutton select. 40

4.30 Block diagram of the sequence method using

pushbutton pre-process 41

4.31 Block diagram of the sequence method using

pushbutton extract 41

4.32 Block diagram of the sequence method using

pushbutton recognize 42

5.1 Setup the adapter using pushbutton video setup 44

5.2 Choose the best pixel 44

5.3 Capture image using pushbutton snap 45

5.4 Video Viewer 45

5.5 Video Viewer 1 45

5.6 Video Viewer 2 46

5.7 Video Viewer 3 46

(15)

xiv

5.9 Training with noise 46

5.10 Alphabet A 47

5.11 Alphabet B 47

5.12 Alphabet C 47

5.13 Alphabet D 48

5.14 Alphabet E 48

5.15 Alphabet F 48

5.16 Alphabet G 49

5.17 Alphabet H 49

5.18 Alphabet I 49

5.19 Alphabet J 50

5.20 Alphabet K 50

5.21 Alphabet L 50

5.22 Alphabet M 51

5.23 Alphabet N 51

5.24 Alphabet O 51

5.25 Alphabet P 52

5.26 Alphabet Q 52

5.27 Alphabet R 52

5.28 Alphabet S 53

5.29 Alphabet V 53

5.30 Alphabet W 53

5.31 Alphabet X 54

5.32 Alphabet Y 54

5.33 Alphabet T 55

5.34 Alphabet U 55

5.35 Alphabet Z 55

5.36 Alphabet 1 56

(16)

5.38 Alphabet 3 56

5.39 Alphabet 4 57

5.40 Alphabet 5 57

5.41 Alphabet 6 57

5.42 Alphabet 7 58

5.43 Alphabet 8 58

5.44 Alphabet 9 58

5.45 Chose image 59

5.46 Load image 59

5.47 Select character 59

5.48 Pre-Process the image 60

5.49 Extract the image using Neural Network 60

5.50 Recognize number 0 60

5.51 Recognize number 1 61

5.52 Recognize number 2 61

5.53 Recognize number 3 61

5.54 Recognize number 4 62

5.55 Recognize number 5 62

5.56 Recognize number 6 62

5.57 Recognize number 7 63

5.58 Recognize number 8 63

(17)

xvi

LIST OF ABBREVIATION

1. NN = Neural Network

2. ANN = Artificial Neural Network 3. GUI = Graphical User Interface

4. LPR = License Plate Recognition System 5. MLP = Multilayer Perceptron

(18)

CHAPTER I

INTRODUCTION

1.0 Introduction

The number of vehicle in metropolitan cities is always growth. The growth has a positive implication to provide a professional and sufficient parking system. This fact requires a plate number recognition system to support a parking arrangement.

The aim of this final project is to develop Simulink Model of License Plate Recognition system using Simulink, Video and Image Processing Block set R2006a (software).

License Pate Recognition system (LPR) is an image-processing technology used to identify vehicles by their license plates number. LPR project is to present a real time car park control system. A video-based car park model will be built by using Simulink of Video and Image Processing Block set to capture the license plate number.

The proposed of this License Plate Recognition system (LPR) works, whenever the vehicle approached the secured area (Entrance). The sensor senses the car and its presence signaled to the LPR unit. After that, the LPR unit activates the illumination (Infrared) and takes pictures of the front plates from the LPR camera. The images of the vehicle include the plate and the pixel information is read by the LPR unit's image processing hardware. Then, the LPR unit checks if the vehicle appears on a predefined list of authorized cars compare with the database, and if found, it gives signals to open the gate. Next, the authorized vehicle enters into the secured area. After passing the gate, its detector closes the gate. Now the system waits for the next vehicle to approach the secured area. The same process will be repeated when the car goes out of the parking lot.

(19)

1.1 Project Objectives

The objective of the project proposal:

1. To develop Simulink Model of License Plate Recognition System using Video and Image Processing Block set (software).

2. Capturing images (license plate number) 3. Training character of the license plate number

4. Recognized the character of the license plate number.

1.2 Problem Statements

Nowadays parking lots are becoming organic part of our everyday life. We use the entrance, day by day when going to work, shop, a hotel, a restaurant, an airport, a hospital etc. But, for people who are living in private area like in condominium, they have to wait for guard to open the gate before cars can enter the entrance. Parking our car, leaving it in safe, finding it there again, fast, comfortably and easily, that's the goal. To reduce time, increase efficiency and security, that's an important issue. License plate recognition system using video and images offers flexible, low maintenance, yet effective and intelligent technology for car park operation: ticketing, security, location services, etc.

With integration of license plate recognition system using video and images into parking management systems in private area, it becomes possible to automate vehicle entry to and exit from a car park or a secure zone and to use the recognized registration number for inventory management. The usage of license plate recognition with specially designed camera set, equipped and synchronized with a Infra Red flash - provides quick and accurate identification of vehicles 24 hours a day, 7 days a week, without human assistance or any driver co-operation. It is a significant added value for comprehensive parking management systems.

(20)

1.3 Scope Project

The scope of this project using Simulink, Video and Image Processing Block set R2006a (software) are:

1. Used the system for private area only, such as in condominium 2. Capture the license plate recognition.

3. Takes picture of the front plate

4. Train the alphabet (A to Z) and number (0 to 9) using neural network.

(21)

CHAPTER II

LITERATURE REVIEW

This chapter reviews existing project created to get an idea about the project design, conception, specification and any information that related to improve the project. In this chapter, some review about an Artificial Neural Network system that proposed to fulfill this project will also be reported.

2.1 Purpose statement for Neural Network approach

Tim Klassen [1] as mentioned in the introduction, this research will show on-line average Arabic character recognition rates above 80% and training recognition rates above 90% using Neural Networks for feature extraction and classification with multiple unconstrained 19 writers. Linear networks will be emphasized, where this represents the lowest computational overhead during both training and recall, hence suitable for the target application device.

2.2 Artificial Neural Network

Haris Al-Qodri Maarif, and Sar Sardy [2] mentioned that Artificial Neural Network (ANN) is a network which imitate to human neural network with some simple processors (units). Each unit has some local memory and connected unidirectional to flow the numerical data. Those units are operated only at local data and input. ANN has some components [3]:

(22)

5 1. Neuron is a unit consists of input, weight, bias, adder and transfer

function.

2. Weight defines the position of hyper plane in input space.

3. Bias is used to change the position of hyper plane in order not to across the zero hyperspace.

4. Transfer function gives a threshold value to activate neuron. Some of transfer functions are illustrated on Fig.2.1.

Figure 2.1:Some graphics of transfer function

2.3 Recognition

To recognize the plate, Artificial Neural Network (ANN) is used. ANN is designed to recognize 26 characters of letter and 10 characters of number. Two ANN topology are designed to recognize letter and number. The designed ANN is a back prop network (supervised learning), which is able to recognize input matrix in size 1326 x n. Designing of ANN is conducted by applying function newff. ANN has three layers consisting of input layer, hidden layer and output layer. For ANN‟s number, there will be 10 neurons for hidden layer and output layer. For ANN‟s letter, there will be 20 and 26 neurons for hidden layer and output layer respectively.

(23)

6 Target for the ANN are 26 elements for ANN‟s letter and 10 elements for ANN‟s number. The value of element of each targets are all zero except one element on specific position which represent the number or letter. For example, the first element is 1 in letter‟s target, represent A. Learning process applied in this network, is using function traingdx with epoch = 3000, MSE = 1 x 10-5, and learning rate = 0.05.

Output from the network will be two dimensional matrixes with size 26x n and 10 x n for ANN‟s letter and ANN‟s number, respectively. The value of output will be in range 0 and 1. To be recognize, output should be processed first by converted the highest value to be 1 and other will be 0. Then, the location of element which has value 1 will be founded and the result will represent number or letter. For example, output from ANN‟s letter which has value 1 at first element will represent the letter A, [2].

2.4 Preprocessing/Image Extraction

During this process we scan the handwritten or printed character from a sheet of paper with the help of a scanning device into graphic/image file format (BMP). As Mr.Danish Nadeem and Miss Salena Rizvi [4], mentioned, the computer images are made up of pixels, which have a specific color associated with each of them. This is identified by the pixel‟s value. These pixels are arranged in the form of a matrix of horizontal rows and columns. Once the image file is scanned, the information about the number of rows and columns in the image, pixel offset, and size of image are contained there in collectively called as the image header. When we are dealing with a 24-bit bitmap the header is stored in the first 54 bytes as explained in the header file (bmp.h).

Each image format stores the pixels and their positions in a specific manner. The 24-bit bitmap uses 3 bytes to store the color (pixel value) associated with each pixel. In BMP files the preprocessing starts from bottom to the top, hence, the color of each pixel is stored starting from the bottom and moving right hand upwards. Once we extract the pixel values associated with each pixel of the image, their

(24)

7 decimal equivalent values are stored in a matrix at the corresponding row column position. The size of this matrix is equal to the (number of rows * no. of columns (of the image)).

Figure 2.2: Extraction of a character in a matrix

The extraction of the image “E” into a corresponding matrix is shown above. Here w= Numerical value corresponding to white and b= Numerical value corresponding to black. From this matrix we find the element which occur the most. This will correspond to the most prominent color, and hence the background in our case. This is very obvious as the character will be less prominent than the background of the paper on which it has been written or printed. Therefore the number of pixels constituting the background will be less than the number of pixels that make up the character. Hence our assumption is that the numerical values other than the background constitute the character

(1)

1.1 Project Objectives

The objective of the project proposal:

1. To develop Simulink Model of License Plate Recognition System using Video and Image Processing Block set (software).

2. Capturing images (license plate number) 3. Training character of the license plate number 4. Recognized the character of the license plate number.

1.2 Problem Statements

(2)

1.3 Scope Project

The scope of this project using Simulink, Video and Image Processing Block set R2006a (software) are:

1. Used the system for private area only, such as in condominium 2. Capture the license plate recognition.

3. Takes picture of the front plate

4. Train the alphabet (A to Z) and number (0 to 9) using neural network.

(3)

CHAPTER II

LITERATURE REVIEW

2.1 Purpose statement for Neural Network approach

2.2 Artificial Neural Network

(4)

1. Neuron is a unit consists of input, weight, bias, adder and transfer function.

2. Weight defines the position of hyper plane in input space.

3. Bias is used to change the position of hyper plane in order not to across the zero hyperspace.

4. Transfer function gives a threshold value to activate neuron. Some of transfer functions are illustrated on Fig.2.1.

Figure 2.1: Some graphics of transfer function

2.3 Recognition

(5)

Target for the ANN are 26 elements for ANN‟s letter and 10 elements for

ANN‟s number. The value of element of each targets are all zero except one element on specific position which represent the number or letter. For example, the first

element is 1 in letter‟s target, represent A. Learning process applied in this network, is using function traingdx with epoch = 3000, MSE = 1 x 10-5, and learning rate = 0.05.

2.4 Preprocessing/Image Extraction

(6)

decimal equivalent values are stored in a matrix at the corresponding row column position. The size of this matrix is equal to the (number of rows * no. of columns (of the image)).

Figure 2.2: Extraction of a character in a matrix

License Plate Recognition System Using Video And Images.