Development Of Image Filtering System For Car Sticker Detection.

(1)

UNIVERSITI TEKNIKAL MALAYSIA MELAKA

DEVELOPMENT OF IMAGE FILTERING SYSTEM FOR CAR

STICKER DETECTION

This report submitted in accordance with requirement of the Universiti Teknikal Malaysia Melaka (UTeM) for the Bachelor Degree of Manufacturing Engineering

(Robotic and Automation) (Hons.)

by

ELYSA SAKINAH BINTI MD. AMIN B050910015

900607-13-7622

FACULTY OF MANUFACTURING ENGINEERING 2013


(2)

** Jika Laporan PSM ini SULIT atau TERHAD, sila lampirkan surat daripada pihak berkuasa/organisasi berkenaan dengan menyatakan sekali sebab dan tempoh laporan PSM ini perlu dikelaskan sebagai SULIT atau TERHAD.

UNIVERSITI TEKNIKAL MALAYSIA MELAKA

BORANG PENGESAHAN STATUS LAPORAN PROJEK SARJANA MUDA TAJUK: DEVELOPMENT OF IMAGE FILTERING SYSTEM FOR CAR STICKER DETECTION

SESI PENGAJIAN: 2012/13 Semester 2

Saya ELYSA SAKINAH BINTI MD. AMIN

mengaku membenarkan Laporan PSM ini disimpan di Perpustakaan Universiti Teknikal Malaysia Melaka (UTeM) dengan syarat-syarat kegunaan seperti berikut: 1. Laporan PSM adalah hak milik Universiti Teknikal Malaysia Melaka dan penulis. 2. Perpustakaan Universiti Teknikal Malaysia Melaka dibenarkan membuat salinan

untuk tujuan pengajian sahaja dengan izin penulis.

3. Perpustakaan dibenarkan membuat salinan laporan PSM ini sebagai bahan pertukaran antara institusi pengajian tinggi.

4. **Sila tandakan () SULIT

TERHAD

TIDAK TERHAD

(Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysiasebagaimana yang termaktub dalam AKTA RAHSIA RASMI 1972)

(Mengandungi maklumat TERHAD yang telah ditentukan oleh organisasi/badan di mana penyelidikan dijalankan)

Alamat Tetap:

No.1341, Lorong D11, Tmn Satria Jaya BDC Stampin, Off Jln Stutong, 93350, Kuching, Sarawak

Tarikh:

Disahkan oleh:

PENYELIA PSM


(3)

DECLARATION

I hereby, declared this report entitled “Development of Image Filtering System for Car Sticker Detection” is the results of my own research except as cited in references.

Signature : ………..

Author’s Name : ELYSA SAKINAH BINTI MD AMIN


(4)

APPROVAL

This report is submitted to the Faculty of Manufacturing Engineering of UTeM as a partial fulfillment of the requirements for the degree of Bachelor of Manufacturing Engineering (Robotics and Automation) (Hons.). The member of the supervisory committee is as follow:

……… Supervisor


(5)

ABSTRAK

Projek ini dihasilkan bagi mengkaji teknik penapisan imej yang boleh digunakan bagi menghasilkan algoritma sistem penapisan imej yang akan digunakan untuk membantu pengesanan pelekat kenderaan di UTeM. Sistem verifikasi yang terkini tidak mampu untuk mengesan imej pelekat yang diambil dalam pelbagai faktor persekitaran dan posisi.Algoritma sistem penapisan ini dihasilkan dengan menggunakan MATLAB 2013a. Antara cara-cara yang digunakan dalam teknik penapisan ini merangkumi ‘Color Correlation Technique’, ‘Accelerated Sum Squared Difference using Cross

Correlation Technique’dan ’Point Match Features using SURF Features Technique’.

Teknik-teknik penapisan imej ini telah diuji dengan menggunakan imej dalam pelbagai jenis kecerahan dan darjah pusingan. Keputusan dari hasil eksperimen ini menunjukkan salah satu daripada teknik tersebut menghasilkan keputusan yang memuaskan berbanding teknik lain.


(6)

ABSTRACT

The purpose of this report is to study a filtering algorithm that could be used to assist the current UTeM’s sticker verification system. The current verification system is not able to detect stickers in images taken under variable environmental behavior and of unfixed position on the windscreen of a car. The filtering algorithm for this project was done by using MATLAB 2013a. The methods that were used in this filtering technique include Color Correlation Technique, Accelerated Sum Squared Difference using Cross Correlation Technique and the Point Match Features using SURF Features Technique. These techniques have been tested using image samples having different range of brightness and different degrees of rotation. The result of the experiment showed one of the technique produced a good performance compared to others.


(7)

DEDICATION

This is devoted to my father who never stops believing and supporting me throughout my journey in building success and also to my mother who teaches me that nothing comes easy in life.


(8)

ACKNOWLEDGEMENT

I would like to thank Encik Mohd Nazrin bin Muhammad for being the best supervisor of the task. His perceptive advice, perceptive criticisms and patient encouragement aided the writing of the report in inestimable ways. His support of the project was greatly needed and deeply appreciated.

Other than that, thousands of gratitude dedicated to UTeM’s staffs and technicians, friends and everyone that had given support throughout the process in completing the project.


(9)

TABLE OF CONTENT

Abstrak i

Abstract ii

Dedication iii

Acknowledgement iv

Table of Content v

List of Tables ix

List of Figures x

List of Abbreviations xi

CHAPTER 1: INTRODUCTION 1

1.1 Image Processing 1

1.1.1 History of Image Processing 2

1.1.2 Advantage of Image Filtering 3

1.2 Image Filtering Applications 4

1.3 Problem Statement 6

1.4 Objectives 7

1.5 Scope 7

1.6 Report Structure 7

CHAPTER 2: LITERATURE REVIEW 9

2.1 Image Processing. 9

2.1.1 License Plate Recognition. 9

2.1.2 Image Enhancement. 11

2.1.3 Image Enhancement for Non-uniform Lighting Conditions. 12 2.1.4 Filtering Technique for Improving Facial Identification. 13


(10)

2.2.1 MATLAB 15

2.2.2 C++ Programming Using OpenCV 15

2.3 Techniques 16

2.3.1 Morphological Process 16

2.3.2 Normalization 17

2.3.3 Edge Enhancement Techniques 18

2.4 Chapter Summary 19

CHAPTER 3: METHODOLOGY 20

3.1 Project Planning 20

3.2 Understanding on the Previous Verification System/ 21

3.3 Problem Tree Analysis 22

3.4 Filtering Techniques for Image Sample Testing 23

3.4.1 Color Correlation Technique 24

3.4.2 Accelerated Sum Squared Difference Using Cross Correlation

(SSDXCORR) 25

3.4.3 Point Feature Matching Using SURF Features Technique 26

3.5 Image Capturing Process/ 28

3.6 Steps of Image Filtering Techniques/ 32

3.6.1 Color Correlation Technique/ 32

3.6.2 Accelerated Sum Squared Difference Using Cross Correlation

(SSDXCORR) Technique/ 34

3.6.3 Point Feature Matching Using SURF Features Technique 35

3.7 Chapter Summary/ 38

CHAPTER 4: RESULTS AND DISCUSSION 39

4.1 Experimental Setup 39


(11)

4.2 Conducting the Experiment 40

4.3 Result of Image Testing 40

4.3.1 Technique 1: Color Correlation Technique 42

4.3.1.1 Results on Samples of Different Range of Brightness 42 4.3.1.2 Results on Samples of Different Degrees of Rotation 43 4.3.2 Technique 2: SSDXCORR (Cross Correlation based Sum of

Squared Difference) 44

4.3.2.1 Results on Samples of Different Range of Brightness 44 4.3.2.2 Results on Samples of Different Degrees of Rotation 45 4.3.3 Technique 3: Point Feature Matching SURF Features Function 46 4.3.3.1 Results on Samples of Different Range of Brightness 46 4.3.3.2 Results on Samples of Different Degrees of Rotation 47

4.4 Discussion 48

4.5 Chapter Summary 49

CHAPTER 5: CONCLUSION AND RECOMMENDATIONS 50

5.1 Conclusions 50

5.2 Recommendations 51

REFERENCE 52

APPENDICES

A Example of coding s used in the project B Gantt Chart of Project


(12)

LIST OF TABLES

3.1 The distance and height for image capturing process. 30 3.2 The different range of brightness and the example of image sample. 31 3.3 The different degrees of rotation and the example of image samples 32 4.1 The division of method according to sample classification 43 4.2 The result of Method 1 and Method 2 tested on image sample of different

range of brightness for the Color Correlation Technique 44 4.3 The result of Method 1 and Method 2 tested on image samples with

different degrees of rotation for the Color Correlation Technique 45 4.4 The result of Method 1 and Method 2 tested on image sample of different

range of brightness for the SSDXCORR Technique 46

4.5 The result of Method 1 and Method 2 tested on image samples with

different degrees of rotation for the SSDXCORR Technique 47 4.6 The result of Method 1 and Method 2 tested on image sample of different

range of brightness for the SURF Features Technique. 48 4.7 The result of Method 1 and Method 2 tested on image samples with


(13)

LIST OF FIGURES

1.1 A digital picture produced in the early 1920’s 2

1.2 Different levels of image processing 3

1.3 Grayscale image before and after filtering process 4

1.4 Edge enhancement applied on MRI image 5

1.5 IC inspection layout for visual inspection 5

1.6 Image of license plate recognition system 6

2.1 Block diagram of plate extraction module (Rajesh et. al, 2010) 10

2.2 Example of plate extraction image 11

2.3 Example of image of The Empire State Building 12

2.4 Example of colored image enhancement 13

2.5 Sub-image homomorphic filtering method (Delac et. al, 2006) 14

2.6 Example of sub-image filtering results 14

2.7 Example of MATLAB practice 15

2.8 Example of OpenCV application for image processing 16

2.9 Example of Digital Image Processing 18

2.10 Example of Histogram Stretching method 19

3.1 General process flow of the project 22

3.2 The problem tree analysis of poor image quality 24

3.3 Example of Color Correlation Technique 26

3.4 Example of Accelerated Sum Squared Difference using Cross Correlation 27 3.5 Example of Point Match Feature Using SURF Features Technique 29 3.6 Example Color Correlation Technique Used in Sticker Detection 34 3.7 Example Accelerated Sum Squared Difference using Cross Correlation

Technique Used in Sticker Detection 36


(14)

LIST OF ABBREVIATIONS, SYMBOLS AND

NOMENCLATURE

OpenCV - Open Source Computer Vision

SSDXCORR - Accelerated Sum Squared Difference using Cross Correlation SURF - Speed-Up Robust Features


(15)

CHAPTER 1

INTRODUCTION

This project is about developing an image filtering algorithm that will be used to assist a sticker verification system for detecting and classifying car stickers. The main object of interest in this project will be the new UTeM sticker available on windscreens of cars. The filtering algorithm will process image by extracting the UTeM sticker based on the similar properties provided by the template image. The extracted sticker will then be used in the subsequent process which will be the current verifying system. This chapter will give details on the history of image processing, the advantages of having a filtering algorithm in an image processing system as well as the applications of image processing. Besides that, the problem statement, objectives and scopes of this project were also stated in this chapter.

1.1 Image Processing

Image processing refers to any processes or operations that function as to improve, correct, analyze or change the image differing to its original state. In other words, it is the manipulation and analysis of pictorial information. Most of the existing methods of


(16)

image processing involve treating an image as a two-dimensional signal and applying standard processing techniques to obtain the best analysis result. In general, the objective of image processing is to transform or analyze an image so that the new information of the image is made obvious. An image processing flow consist of enhancement process, restoration process, analysis, compression process and synthesizing.

1.1.1 History of Image Processing

Image processing was firstly used back in the early 1920’s where it was applied in the newspaper industry. In those years, the Bartlane Cable System has been used to transfer images through cable lines. This system was invented by Harry G. Bartholomew and Maynard D. McFarlane; whom succeeded in transferring the first picture across Atlanta by submarine cable between London and New York.

Figure 1.1: A digital picture produced in 1921 from a coded tape by a telegraph printer with special type faces, (McFarlane, 1972).

Starting from the mid of the 1920’s, improvements were made to the system which result in image of higher quality. After that, in the 1960’s more improvements were done due to technology development which enable the processing of an image of the moon taken by the Ranger 7 probes. Image processing was also applied in medical application later in the next decade where Sir Godfrey N. Hounsfield and Professor


(17)

Allan M. Cormack invented the Tomography, the technology behind Computerized Axial Tomography scan.

1.1.2 Advantages of Image Filtering

Image filtering is the first level in an image processing flow which is also known as the Low Level Process. This level of processing is used to enhance an image or video by eliminating noise, preserving edges and removing unwanted data. Pictorial information can be preserved or improved for ease of human interpretation or storage and transformation as well as for autonomous machine perception. Figure 1.2 shows the different category of image processing.

Figure 1.2: The different levels of image processing.

The advantages of the image filtering process include improving the quality of the original image by eliminating noise, preserving edges and removing inaccuracies in the image’s data. Besides that, this process can also be used to extract desired region or object in an image. Without a filtering process, the subsequent image analysis operation such as the object recognition or segmentation might not be able to perform well due to its inability to identify the features in unfiltered image. Such example of an image filtering process is an objective image enhancement, which is used in correcting image

Low Level

process

• noise elimination • image sharpening

Mid Level

Process

• object recognition • segmentation

High Level

Process

• scene understanding • autonomous navigation


(18)

for known degradations where it is applied based on known distortions to the original image. Figure 1.3 below shows an example of a grayscale image that has undergone a filtering process of removing noise by using MATLAB.

Figure 1.3: A grayscale image before and after a filtering process.

1.2 Image Filtering Applications

Image filtering is widely applied in many fields such as for medical visualization, industrial inspections, artistic effects, security surveillance and law enforcement as well as for human computer interfaces. In the medical field, image processing is used in Gamma-Ray and X-Ray imaging to detect internal damages in the body that could not be seen or detected by human’s naked eye. In addition to that, the examples of the usage of image filtering are usually used to improve quality on an X-ray and also used in MRI scanning to find the boundaries between types of tissues by edge detection.


(19)

(a) (b)

Figure 1.4: (a) Original MRI image of a dog’s heart. (b) Edge detection image.

Figure 1.4 above shows an example of image filtering done on an MRI image of a dog’s heart. The enhancement process is to improve the visibility features of the original image by applying edge enhancement process.

Besides that, image processing is also applied in the industry to improve the quality of inspection method by improving overall process accuracies and reliability. For example, a vision system that monitors the conveyor can detect bottles that are not filled to its required volume by comparing the captured image with a data stored.

Figure 1.5: IC chips inspection on a conveyer.

Figure 1.5 above shows an example of IC chips inspection layout. The image captured will be compared with data stored to detect any defects on the finished products.


(20)

Furthermore, other applications such as automated license plate reading also requires image filtering to process image captured for traffic monitoring, detection of stolen vehicles or any statistical research. Filtering process is used to identify in which area it has to focus in order to detect the plate numbers. This type of application and other types of character or shape recognizing are used for traffic monitoring and surveillance.

Figure 1.6: Image of a license plate recognition system. (Matas J., Zimmerman K, 2006)

Figure 1.6 above shows an example of image processing applied in license plated detection where the filtering method will identify the region of interest and filter the characters on the plate for identification process.

1.3 Problem Statement

This image filtering project is developed in order to assist the current UTeM car sticker verifying system. The current verifying system is only able to detect stickers in images taken from a static position; captured under environmentally-controlled condition. Thus, the system does not have the ability to process images captured under uncontrolled environment such as in changing of lighting or and different object orientation due to the


(21)

absence of image filtering process in the system. This disadvantage causes the verifying system to be unable to perform the verifying process when no sticker is able to be detected.

In contrary, the availability of an image filtering algorithm will eventually improve the verifying system’s ability in processing image containing the UTeM car sticker and hence improving the reliability of the sticker verification system at the entrance of the university.

1.4 Objectives

1. To investigate the current image filtering techniques. 2. To develop an image filtering algorithm using MATLAB. 3. To assess the performance of the algorithm.

1.5 Scope

The scope of this project is limited to capturing images of cars; focusing on the windscreen region. The images will be captured under variable lighting condition and the surrounding environment is not fixed in order to overcome the problem of the image being affected by the change of environment’s behavior. The image captured will be used as the image samples. Furthermore, the project is focusing on extracting the sticker from a target image in order to be used in the subsequent process.

1.6 Report Structure

The report of this project began with three main chapters which are the introduction, literature review and methodology. The first chapter comprises of the introduction towards image processing, the problem statement of the project, the objectives and also


(22)

the scope of this project. The introduction of the image processing gives basic information about the definition of image processing, the levels of image processing and the examples of the application of image filtering. Besides that, this chapter also explains about the problems that arise and needed to be solved. After that, the objectives were stated and were followed by the explanation of the scopes of the project.

In Chapter 2, several references on previous research on the application of image filtering was explained. In addition to that, several explanation on the types of image filtering techniques were also included.

Next, Chapter 3 explains about the method used in this project, from the beginning to the end of obtaining the results of the experiments done.

Chapter 4 comprises of the results and the brief explanations on the experiments are shown in the result section while the explanations about the image filtering functions regarding the experiments are shown in the discussion section. Finally, Chapter 5 will be the conclusion and future works of the project.


(23)

CHAPTER 2

LITERATURE REVIEW

2.1 Image Processing

2.1.1License Plate Recognition

Paper 1: Extraction of License Plate Recognition in Automation License Plate Recognition

The purpose of license plate recognition system is to keep an image record for vehicle identification in public areas or in any premises. License plates may be made of different materials, composition and reflectivity (Rajesh, 2010).The paper presents an image extraction by using the concept connected features in the image. The characters of the license plate are possible to be separated from the rest of the features in the image due to its unique size or arrangement. Hence the research proposed a method in exploiting the features of the license plate enabling it to be extracted. The proposed method uses a software module which will extract the potential region of the license plate by using the connected component concept. The features of the image in the extracted region will be


(24)

filtered and cropped, and at the same time the cropping process also reduces noise in the image. Only then the image will be converted to a gray scale image and then to a binary type.

Figure 2.1: Block diagram of plate extraction module (Rajesh et. al, 2010).

The block diagram shown in Figure 2.1 is of the plate extraction module of the proposed method. The colored image will first be converted into gray scale before being cropped into specified size. After that, the cropped image will be converted into binary and the plate number region will be extracted when desired characters has been recognized. Lastly the result of plate region with noise will be analyzed in further processes. Some of image examples are shown in Figure 2.2.

RGB IMAGE CROPPED GRAYSCALE IMAGE

BINARY IMAGE

PLATE REGION EXTRACTION SUBTRACT FROM ORIGINAL BINARY

IMAGE PLATE REGION


(1)

(a) (b)

Figure 1.4: (a) Original MRI image of a dog’s heart. (b) Edge detection image.

Figure 1.4 above shows an example of image filtering done on an MRI image of a dog’s heart. The enhancement process is to improve the visibility features of the original image by applying edge enhancement process.

Besides that, image processing is also applied in the industry to improve the quality of inspection method by improving overall process accuracies and reliability. For example, a vision system that monitors the conveyor can detect bottles that are not filled to its required volume by comparing the captured image with a data stored.

Figure 1.5: IC chips inspection on a conveyer.

Figure 1.5 above shows an example of IC chips inspection layout. The image captured will be compared with data stored to detect any defects on the finished products.


(2)

Furthermore, other applications such as automated license plate reading also requires image filtering to process image captured for traffic monitoring, detection of stolen vehicles or any statistical research. Filtering process is used to identify in which area it has to focus in order to detect the plate numbers. This type of application and other types of character or shape recognizing are used for traffic monitoring and surveillance.

Figure 1.6: Image of a license plate recognition system. (Matas J., Zimmerman K, 2006)

Figure 1.6 above shows an example of image processing applied in license plated detection where the filtering method will identify the region of interest and filter the characters on the plate for identification process.

1.3 Problem Statement

This image filtering project is developed in order to assist the current UTeM car sticker verifying system. The current verifying system is only able to detect stickers in images taken from a static position; captured under environmentally-controlled condition. Thus, the system does not have the ability to process images captured under uncontrolled environment such as in changing of lighting or and different object orientation due to the


(3)

absence of image filtering process in the system. This disadvantage causes the verifying system to be unable to perform the verifying process when no sticker is able to be detected.

In contrary, the availability of an image filtering algorithm will eventually improve the verifying system’s ability in processing image containing the UTeM car sticker and hence improving the reliability of the sticker verification system at the entrance of the university.

1.4 Objectives

1. To investigate the current image filtering techniques. 2. To develop an image filtering algorithm using MATLAB. 3. To assess the performance of the algorithm.

1.5 Scope

The scope of this project is limited to capturing images of cars; focusing on the windscreen region. The images will be captured under variable lighting condition and the surrounding environment is not fixed in order to overcome the problem of the image being affected by the change of environment’s behavior. The image captured will be used as the image samples. Furthermore, the project is focusing on extracting the sticker from a target image in order to be used in the subsequent process.

1.6 Report Structure

The report of this project began with three main chapters which are the introduction, literature review and methodology. The first chapter comprises of the introduction towards image processing, the problem statement of the project, the objectives and also


(4)

the scope of this project. The introduction of the image processing gives basic information about the definition of image processing, the levels of image processing and the examples of the application of image filtering. Besides that, this chapter also explains about the problems that arise and needed to be solved. After that, the objectives were stated and were followed by the explanation of the scopes of the project.

In Chapter 2, several references on previous research on the application of image filtering was explained. In addition to that, several explanation on the types of image filtering techniques were also included.

Next, Chapter 3 explains about the method used in this project, from the beginning to the end of obtaining the results of the experiments done.

Chapter 4 comprises of the results and the brief explanations on the experiments are shown in the result section while the explanations about the image filtering functions regarding the experiments are shown in the discussion section. Finally, Chapter 5 will be the conclusion and future works of the project.


(5)

CHAPTER 2

LITERATURE REVIEW

2.1 Image Processing

2.1.1License Plate Recognition

Paper 1: Extraction of License Plate Recognition in Automation License Plate Recognition

The purpose of license plate recognition system is to keep an image record for vehicle identification in public areas or in any premises. License plates may be made of different materials, composition and reflectivity (Rajesh, 2010).The paper presents an image extraction by using the concept connected features in the image. The characters of the license plate are possible to be separated from the rest of the features in the image due to its unique size or arrangement. Hence the research proposed a method in exploiting the features of the license plate enabling it to be extracted. The proposed method uses a software module which will extract the potential region of the license plate by using the connected component concept. The features of the image in the extracted region will be


(6)

filtered and cropped, and at the same time the cropping process also reduces noise in the image. Only then the image will be converted to a gray scale image and then to a binary type.

Figure 2.1: Block diagram of plate extraction module (Rajesh et. al, 2010).

The block diagram shown in Figure 2.1 is of the plate extraction module of the proposed method. The colored image will first be converted into gray scale before being cropped into specified size. After that, the cropped image will be converted into binary and the plate number region will be extracted when desired characters has been recognized. Lastly the result of plate region with noise will be analyzed in further processes. Some of image examples are shown in Figure 2.2.

RGB IMAGE CROPPED GRAYSCALE IMAGE

BINARY IMAGE

PLATE REGION EXTRACTION SUBTRACT FROM ORIGINAL BINARY

IMAGE PLATE REGION