Development Of Automatic Quality Control System Using Vision System.
UNIVERSITY TEKNIKAL MALAYSIA MELAKA
DEVELOPMENT OF AUTOMATIC QUALITY CONTROL
SYSTEM USING VISION SYSTEM
This report submitted in accordance with requirement of the Universiti Teknikal
Malaysia Melaka (UTeM) for the Bachelor Degree of Manufacturing Engineering
(Robotics and Automation) with honours
by
CHIN GEE SING
B050710006
FACULTY OF MANUFACTURING ENGINEERING
2011
DECLARATION
I hereby, declared this report entitled “Development of Automatic Quality Control
System Using Vision System” is the results of my own research except as cited in
references.
Signature
:
Author’s Name
:
CHIN GEE SING
Date
:
18 MAY 2011
APPROVAL
This report is submitted to the Faculty of Manufacturing Engineering of UTeM as a
partial fulfillment of the requirements for the degree of Bachelor of Manufacturing
Engineering (Robotics and Automation) with honours. The member of the supervisor
committee is as follow:
……………………………………….
Supervisor
ABSTRAK
Mesin sistem visi banyak digunakan dalam industri untuk tujuan pemeriksaan.
Sistem menyusun automatik dan sistem pemeriksaan visi yang dilaksanakan untuk
integrasi telah membina sistem automasi. Kecekapan ia memeriksa dan menyusun
secara teratur telah member banyak faedah untuk mengatasi kelemahan sistem
konvensional yang dijalankan oleh pekerja. Integrasi konveyor ke sistem yang
dicadangkan adalah untuk menghantarkan objek untuk diperiksa. Kemudian, sistem
pemeriksaan dimulakan dari pengesanan sensor fotoelektrik untuk mengaktifkan
“webcam” menangkap gambar melalui satu PLC. Gambar yang ditangkap akan
didigitalkan oleh “webcam” dan menghantar ke hos komputer untuk pemprosesan
gambar dengan menggunakan perisian Vision Builder Automated Inspection (VBAI).
Konfigurasi untuk perisian VBAI akan dipelajarikan sebelum pemeriksaan visi
dijalankan. Sementara itu, grafik antara muka pengguna direkakan dengan
menggunakan
perisian
Labview
tujuan
untuk
pengguna
mengawasi
and
mengawalkan sistem tersebut. Objek yang diperiksa akan disusun secara teratur oleh
robot. Ia adalah satu robot enam paksi yang mempunyai ruang kerja yang berbentuk
separuh-bola. Namun, integrasi elektronik seperti papan “relay” adalah diperlukan
untuk menyelesaikan masalah ketidaksamaan voltan antara PLC dan prosesor Rhino.
Sebagai kesimpulan, projek ini telah dijalankan untuk mengklasifikasikan objek
dengan menggunakan pemeriksaan secara visual dan eksperimental validasi ke atas
prestasi sebenarnya.
i
ABSTRACT
Machine vision systems are widely used in manufacturing industries for inspection
purpose. The implementation of automated sorting system to machine vision system
has develops an integration of automation system. Its efficiency to inspect parts and
sort according to order offers advantages which against the limitation of conventional
human-operated system. The integration of a conveyor to the proposed system feeds
the inspected parts for inspection purposes. Then, the inspection system is started
from a detection of photoelectric sensor to trigger USB webcam by image acquisition
via a Programmable Logic Controller (PLC). The webcam digitized the image data in
order to send to host computer for image processing by using Vision Builder
Automated Inspection (VBAI) software. The configuration of VBAI is studied on a
personal computer before vision inspection is carried out. On the other hand, a
graphical user interface (GUI) is designed by using Labview for person-in-charge to
monitor and control the entire system. After classification from VBAI software, the
inspected parts are arranged accordingly by Rhino robot. It is a six axis robot arm
which consists of a hemisphere of work space. Though, integration of electronic
hardware such as relay boards are necessary to solve the different voltage signal
between PLC and Rhino controller. As a result, this project is performed to classify
parts by using developed visual-based inspection and experimental validated its
actual performance.
ii
ACKNOWLEDGEMENT
I would like to thank for peoples who guided and contributed in various ways to this
PSM I report. This report would not have been possible without the expert advice
and suggestion of my PSM supervisor, Dr. Zamberi bin Jamaludin. Thank you Dr.
Zamberi for the invaluable information and suggestions that you provided and your
willingness for contribute your precious time to this effort. Besides that, I also thanks
to laboratory technician, En. Faizul and Pn. Norhafisah for giving me technical
support.
Thanks also to the Universiti Teknikal Malaysia Melaka (UTeM) for offering PSM
as a compulsory subject for students to gain skills on organizing a project.
Beyond the professional involved in this report, my coursemate and family were also
quite influential in this work. Thanks to my coursemate for sharing their precious
opinion, experience, comments and resources. Thank you for being there to listen to
my gripe about my work. Thanks for my beloved family unending support and
motivation.
Finally, thanks for the peoples who were there to lend a hand along the way. Thank
you.
iii
DEDICATION
I dedicate this report to my beloved parents, without their patience, support
understanding and most of all loves, the completion of work would have not been
possible.
iv
TABLE OF CONTENT
Abstract
i
Abstrak
ii
Dedication
iii
Acknowledgement
iv
Table of Content
v
List of Tables
ix
List of Figures
x
List Abbreviations
xiii
1. INTRODUCTION
1
1.1
Background
1
1.2
Problem Statement
2
1.3
Objectives
2
1.4
Scope
3
1.5
Expected Result
3
1.6
Content
3
2. LITERATURE REVIEW
5
2.1
Introduction
5
2.2
Vision System
5
2.3
Machine Vision Component
7
2.3.1
Vision Sensor Types
9
2.3.1.1
CCD
9
2.3.1.2
CMOS
14
2.3.1.3
Comparison between CCD and CMOS
17
2.3.2
Image Processing Technique
18
2.3.2.1
Smoothing Technique
20
2.3.2.1.1 Lowpass Filter
20
2.3.2.1.2 Local Average
21
2.3.2.1.3 Gaussian Filter
21
v
2.3.2.1.4 Median Filter
21
2.3.2.2
22
Edge Detection Technique
2.3.2.2.1 Prewitt Operator
23
2.3.2.2.2 Sobel Operator
24
2.3.2.2.3 Roberts Operator
24
2.4
Sorting System
25
2.4.1
Robot
26
2.4.2
Types of Robot Manipulator
27
2.5
Conveyor
29
2.5.1
Traditional Conveyor
30
2.5.2
Evolution of Conveyor
30
2.6
Measurement Devices
31
2.6.1
Analogue-to-Digital Converter
32
2.6.2
Photoelectric Sensor
33
2.7
Programmable Logic Controller (PLC)
36
2.8
Summary
38
3. METHODOLOGY
39
3.1
Introduction
39
3.2
Overall Research Strategies
39
3.3
Project Structure
43
3.3.1
Image Processing Procedures
45
3.4
Characterization of System Components
46
3.4.1
Conveyor
47
3.4.2
Rhino Robot Arm
48
3.4.3
Webcam
49
3.4.4
Personal Computer
49
3.4.5
Photoelectric Sensor
50
3.4.6
Programmable Logic Controller (PLC)
52
3.5
Summary
52
4. EXPERIMENTAL SETUP AND PROCEDURES
55
4.1
55
Introduction
vi
4.2
Experimental Setup
55
4.2.1
Hardware Component
55
4.2.2
Software Component
58
4.2.2.1
Logic of Electrical Control System
58
4.2.2.2
PC Access
60
4.2.2.3
Graphical User Interface (GUI)
62
4.2.2.4
Image Processing Configuration
63
4.2.2.5
Pick and Place Configuration
66
4.3
Experimental Setup Procedures
67
4.4
Summary
68
5. RESULT AND DISCUSSION
69
5.1
Introduction
69
5.2
Results
69
5.2.1
Ladder Diagram
70
5.2.2
Development of I/O Server
71
5.2.3
Graphical User Interface Design and Data Binding
72
5.2.4
Vision Inspection
74
5.2.5
Sorting
75
5.3
Discussion
75
5.3.1
Circuit Board for Relay
76
5.3.2
Matched Timing and Sequence between Robot Arm
77
and Image Acquisition
5.3.3
Numeric Indicators
77
5.4
Suggestion
78
5.4.1
Robot Feedback
78
5.4.2
Robot Efficiency
79
5.5
Summary
79
6. CONCLUSION
80
REFERENCES
82
vii
APPENDICES
85
A
Ladder Diagram
86
B
Rhino Robot Programming
90
C
Block Diagram of GUI
92
viii
LIST OF TABLES
2.1
Feature and Performance Comparison of CCD and
18
CMOS
2.2
Correction Factor for Direct Reflective Photoelectric
35
Sensor
3.1
Gantt Chart for FYP I
53
3.2
Gantt Chart for FYP II
54
4.1
Step for PLC Configuration
59
4.2
Configuration of PC Access
61
4.3
GUI Configuration of Labview
62
4.4
Image Processing Configuration on VBAI
64
4.5
Rhino Robot Configuration using RoboTalkWindow
66
5.1
Condition for Combination of Colour and Shape
70
ix
LIST OF FIGURES
2.1
Basic Functions of Machine Vision System
6
2.2
The Coordinate Convention of Pixels That Represent Digital
7
Images
8
2.3
The Basic Components of a Vision System
2.4
The Example of CCD Image Sensor
10
2.5(a)
CCD Pixel
12
2.5(b)
Architecture
12
2.6
Basic Principle of The Charge-Coupled Device
13
2.7
CMOS Sensor is Implemented in Low-End Web-Cam
14
2.8
CMOS Vision Sensor Floorplan
15
2.9
Schematic of The RGB Colour Cube
19
2.10
The Level of Histogram of Gray Scales
19
2.11
Calculating The Median Value of a Pixel Neighbourhood
22
2.12
A 3 × 3 Region Mask
23
2.13
Prewitt Horizontal and Vertical Operators
23
2.14
Sobel Convolution Kernels
24
2.15
The Computation Method of a Pixel
24
2.16
Roberts Cross Convolution Masks
25
2.17
The Closed-Loop Control System
27
2.18
Cartesian Robot
27
2.19
Cylindrical Robot
28
2.20
Spherical Robot
28
2.21
Articulated Robot
29
2.22
Traditional Conveyor
30
2.23
Steps in Analog-to-Digital Conversion of Continuous Analog
32
Signals from Process
2.24
Thru-Beam Photoelectric Sensor Detection
34
2.25
A Reflection with Reflector Photoelectric Sensor Detection
34
Method
2.26
A Direct Reflection Photoelectric Sensor Detection Method
x
35
2.27
Siemens Programmable Logic Controller, S7-200
36
2.28
Components of a PLC
37
3.1
Overall Work Flow Chart
41
3.2
The Flow Chart Illustrates The Data Collection and Data Study
42
3.3
The Flow Chart of Proposed System
44
3.4
Image Processing Flow Chart
46
3.5
The Schematic Diagram of Integrated System
47
3.6
A Belt Conveyor
47
3.7
Rhino Robot Arm
48
3.8
A Home-Used Webcam
49
3.9
A Personal Computer
50
3.10
Photoelectric Sensor
51
3.11
Siemens S7-200 PLC
52
4.1
CAD Drawing for Webcam Attachment on Conveyer
56
4.2
Actual Frame Mounting on Conveyor
56
4.3
Designed Logic Circuit Board
57
4.4
Overall Experimental Setup
68
5.1
Actual Sorting Result That is Performed by Rhino Robot
70
5.2
Designed Ladder Diagram
71
5.3
Develops and I/O Server Module using PC Access
72
5.4
GUI of The Integrated System
73
5.5
Select The Source Item in Data Binding
73
5.6
State Program for Vision Inspection
74
5.7
Inspection Step for State: Circle Red: x1
75
5.8
Relay Circuit Board Used to Triggered PLC and Motor of
76
Conveyor
5.9
Relay Circuit Board Used to Trigger Robot from PLC Input
77
Signal
5.10
Numeric Indicators for Accept and Reject Counting
xi
78
A1
Ladder Diagram
85
A2
Ladder Diagram
86
A3
Ladder Diagram
87
A4
Ladder Diagram
88
A5
Ladder Diagram
89
C1
Block Diagram for GUI
92
xii
LIST OF ABBREVIATIONS
ADC
-
Analogue-to-Digital Converter
APS
-
Active Pixels
CAPS
-
Complementary Active Pixels Sensors
CCD
-
Charged-Couple Device
CCTV
-
Close Circuit Television
CID
-
Charge-Injection Devices
CMOS
-
Complementary Metal-Oxide Semiconductor
DOF
-
Degree of Freedom
GUI
-
Graphical User Interface
I/O
-
Input/Output
LED
-
Light Emitter Diode
OLE
-
Object Linking and Embedding
OPC
-
OLE for Process Control
PC
-
Personal Computer
PCB
-
Printed Circuit Board
PIC
-
Person-in-charge
PLC
-
Programmable Logic Controller
PMOS
-
P-type metal-oxide-semiconductor
PPP
-
Prismatic, Prismatic, Prismatic
PPS
-
Passive Pixels
PVC
-
Polyvinyl Chloride
RGB
-
Red, Green, Blue
RPP
-
Rotary, Prismatic, Prismatic
ROI
-
Region of Interest
RRP
-
Rotary, Rotary, Prismatic
RRR
-
Rotary, Rotary, Rotary
USB
-
Universal Serial Bus
VBAI
-
Vision Builder Automated Inspection
VDC
-
Voltage Direct Current
VLSI
-
Very-large-scale integration
xiii
2-D
-
Two dimensions
3-D
-
Three dimensions
xiv
CHAPTER 1
INTRODUCTION
1.1
Background
The essential goals in the implementation of automation system in the industries are
to provide improvement in human operator variances and operational efficiency. It is
known in that many cases, product quality consistency can vary depending on the
characteristic of human operators who are on duty. Hence, automation system should
be provided to improve this variance. The problem associated with human-assisted
operation is characterized by cycle time, repeatability and material handling.
Although operational efficiency has been given a top priority, human operators are
still exposed to high probability of injuries, long-term similar working scope in fast
motion, applications naked eyes inspection-based system and insufficient break time.
In order to improve operational efficiency and to improve safety of human operators,
there is a need for efficient, high repeatability and cost effective human-assisted
operation equipment. Mechanized and automized solution that are properly sized
with suitable modularized structure can greatly improve the safety of personnel as
well as efficiency and quality of production. In addition, such efficient automation
system with high repeatability can greatly improve the operational efficiency,
product quality and product consistency when applied in combination with intelligent
instrumentation and advanced control.
The mechanized and automized operation system include robot arm, programmed
conveyor system, inspection system and host computer are necessitated to implement
in industries purposely to increase operational efficiency and product quality. A 6
degree of freedom (D.O.F) of robot arm which possess wide working area has
1
enlarge the approaching region and various orientation. Therefore, it is a wise
solution to be used to substitute the folklore operation system like human operator in
industries by robot arm. Besides, the evolution of conveyor from manual operate to
automated conveyor system has enhanced the product handling and increase the
cycle time as well. The implementation of vision system for inspection in production
line becomes a booster for inspection compare to old-fashion naked eye inspection.
The mechanized and automized systems are integrated with intelligent vision system
enable the whole production processes execute in fully automatic. Thus, the
operational efficiency is increase effectively and greatly decreases the variances of
product quality.
1.2
Problem Statement
Nowadays, human-operator based industries require longer time to inspect using
naked eye and sort the products into the correct slot or container. This action of
inspection and sorting consume more time and results in the operators unable to meet
the cycle time. Low repeatability is one the effects of long working time on similar,
repeating working scope at fast motion. Therefore, the operators may get tired and
bored resulting in bad inspection and sorting quality of the products. Due to the
limitations, an automated inspection system based on a vision system integrated into
a pick-and-place robot is required.
1.3
Objectives
Based on the problem statements stated above, several objectives are to be obtained
at the end of this project. The objectives of this project are:
i.
To design visual-based inspection system
ii.
To classify and sort products using vision inspection system
2
1.4
Scope
This project will focus on vision-based inspection and products sorting. The scopes
of this project are:
i.
Image acquisition using a webcam.
ii.
Configure vision inspection algorithm based on Vision Builder Automated
Inspection (VBAI).
iii.
Validate vision inspection system by interfacing a pick-and-place robot, a
webcam, a personal computer, a programmable logic controller (PLC), and a
conveyor.
iv.
Classify product in term of different colours and shapes.
The fabrication of conveyor, robot arm and development of software will not be
covered in this project.
1.5
Expected Result
This project is expected to improve the inspection process in terms of colour sorting
and shape orientation for packaging purpose. The product will be conveyed and
inspected for its colour and differentiated in term of shape using image processing
technique. A robot then removes the product and places it in a correct slot at right
orientation.
1.6
Content
This report is presented in several chapters that describe the development of
automatic quality control system using vision system. Chapter 2 discusses the
Literature Review that describes about the related information from journals and
books. Chapter 3 describes the Methodology of the proposed work and shows a flow
chart of process that explains the operational process supported with some statement.
3
Chapter 4 explains the Experimental Setup and Procedures of the installation and
integration of vision-based inspection system and sorting system. Chapter 5
illustrates on Results and Discussion about the results obtained from experiment. The
important findings will be presented in a comprehensive manner. A comparison
between the desired result and the actual result are performed in order to identify any
limitations to the system that is being proposed. In addition, the projected objectives
will be reviewed to ensure that goals met. Furthermore, suitable recommendation for
further improvement to the system will be included. Chapter 6 will conclude the
entire project with summarize the main findings and review the scopes covered.
4
CHAPTER 2
LITERATURE REVIEW
2.1
Introduction
This chapter discusses the literature review related to the proposed vision-based
inspection system. Review of previously developed systems in literature review are
discussed and illustrated. This chapter discusses mainly the vision system, sorting
system, conveyor and sensors that are relevant to the development of the proposed
system.
2.2
Vision System
Vision system can be described as image acquisition, followed by image processing
and interpretation. The processed image data is transferred by computer for some
other useful application. Vision system is classified into two dimensional (2-D) and
three dimensional (3-D). 2-D is most suitable for industrial applications that include
dimension measurement, gauging, check for the presence of components and etc.
(Groover, 2000). Vision systems are widely used in inspection section in industry.
The implementation of vision system increase the efficiency and accuracy of
inspection process because the high repeatability and accuracy of machine has
overcome the disadvantages of human-based operation system. Thus, sales of 2-D
vision system outnumber those of 3-D systems by more than ten to one. Figure 2.1
shows the basic functions of machine vision system.
5
Figure 2.1: Basic functions of machine vision system (Groover, 2000).
Image data acquisition is accomplished using a video camera where the image data is
transferred to digitizing system to store the data for further analysis. The camera is
focused on the region of interest (ROI), and an image data is obtained by dividing the
viewing area into a matrix pixel, arranged in pixel arrays. The intensity value of a
pixel is converted by analog-to-digital converter (ADC) to equivalent digital value
which is proportional to the light intensity. An image is represented in continuous
with respect to the x- and y- coordinates and in amplitude as well. In order to convert
it to digital form, an image is needed to digitizing the coordinate value (sampling)
and digitizing the amplitude value (quantization). In practice, the arrangement of
sensor used will determines the method of sampling to generate the image (Gonzalez
and Woods, 2002). The result of sampling and quantization of an image function, f(x,
y) now become discrete quantities resulting in a digital image with M rows and N
columns based on the matrix shown in equation 2.1. The coordinate convention of
pixels that represent digital images is shown in figure 2.2.
6
DEVELOPMENT OF AUTOMATIC QUALITY CONTROL
SYSTEM USING VISION SYSTEM
This report submitted in accordance with requirement of the Universiti Teknikal
Malaysia Melaka (UTeM) for the Bachelor Degree of Manufacturing Engineering
(Robotics and Automation) with honours
by
CHIN GEE SING
B050710006
FACULTY OF MANUFACTURING ENGINEERING
2011
DECLARATION
I hereby, declared this report entitled “Development of Automatic Quality Control
System Using Vision System” is the results of my own research except as cited in
references.
Signature
:
Author’s Name
:
CHIN GEE SING
Date
:
18 MAY 2011
APPROVAL
This report is submitted to the Faculty of Manufacturing Engineering of UTeM as a
partial fulfillment of the requirements for the degree of Bachelor of Manufacturing
Engineering (Robotics and Automation) with honours. The member of the supervisor
committee is as follow:
……………………………………….
Supervisor
ABSTRAK
Mesin sistem visi banyak digunakan dalam industri untuk tujuan pemeriksaan.
Sistem menyusun automatik dan sistem pemeriksaan visi yang dilaksanakan untuk
integrasi telah membina sistem automasi. Kecekapan ia memeriksa dan menyusun
secara teratur telah member banyak faedah untuk mengatasi kelemahan sistem
konvensional yang dijalankan oleh pekerja. Integrasi konveyor ke sistem yang
dicadangkan adalah untuk menghantarkan objek untuk diperiksa. Kemudian, sistem
pemeriksaan dimulakan dari pengesanan sensor fotoelektrik untuk mengaktifkan
“webcam” menangkap gambar melalui satu PLC. Gambar yang ditangkap akan
didigitalkan oleh “webcam” dan menghantar ke hos komputer untuk pemprosesan
gambar dengan menggunakan perisian Vision Builder Automated Inspection (VBAI).
Konfigurasi untuk perisian VBAI akan dipelajarikan sebelum pemeriksaan visi
dijalankan. Sementara itu, grafik antara muka pengguna direkakan dengan
menggunakan
perisian
Labview
tujuan
untuk
pengguna
mengawasi
and
mengawalkan sistem tersebut. Objek yang diperiksa akan disusun secara teratur oleh
robot. Ia adalah satu robot enam paksi yang mempunyai ruang kerja yang berbentuk
separuh-bola. Namun, integrasi elektronik seperti papan “relay” adalah diperlukan
untuk menyelesaikan masalah ketidaksamaan voltan antara PLC dan prosesor Rhino.
Sebagai kesimpulan, projek ini telah dijalankan untuk mengklasifikasikan objek
dengan menggunakan pemeriksaan secara visual dan eksperimental validasi ke atas
prestasi sebenarnya.
i
ABSTRACT
Machine vision systems are widely used in manufacturing industries for inspection
purpose. The implementation of automated sorting system to machine vision system
has develops an integration of automation system. Its efficiency to inspect parts and
sort according to order offers advantages which against the limitation of conventional
human-operated system. The integration of a conveyor to the proposed system feeds
the inspected parts for inspection purposes. Then, the inspection system is started
from a detection of photoelectric sensor to trigger USB webcam by image acquisition
via a Programmable Logic Controller (PLC). The webcam digitized the image data in
order to send to host computer for image processing by using Vision Builder
Automated Inspection (VBAI) software. The configuration of VBAI is studied on a
personal computer before vision inspection is carried out. On the other hand, a
graphical user interface (GUI) is designed by using Labview for person-in-charge to
monitor and control the entire system. After classification from VBAI software, the
inspected parts are arranged accordingly by Rhino robot. It is a six axis robot arm
which consists of a hemisphere of work space. Though, integration of electronic
hardware such as relay boards are necessary to solve the different voltage signal
between PLC and Rhino controller. As a result, this project is performed to classify
parts by using developed visual-based inspection and experimental validated its
actual performance.
ii
ACKNOWLEDGEMENT
I would like to thank for peoples who guided and contributed in various ways to this
PSM I report. This report would not have been possible without the expert advice
and suggestion of my PSM supervisor, Dr. Zamberi bin Jamaludin. Thank you Dr.
Zamberi for the invaluable information and suggestions that you provided and your
willingness for contribute your precious time to this effort. Besides that, I also thanks
to laboratory technician, En. Faizul and Pn. Norhafisah for giving me technical
support.
Thanks also to the Universiti Teknikal Malaysia Melaka (UTeM) for offering PSM
as a compulsory subject for students to gain skills on organizing a project.
Beyond the professional involved in this report, my coursemate and family were also
quite influential in this work. Thanks to my coursemate for sharing their precious
opinion, experience, comments and resources. Thank you for being there to listen to
my gripe about my work. Thanks for my beloved family unending support and
motivation.
Finally, thanks for the peoples who were there to lend a hand along the way. Thank
you.
iii
DEDICATION
I dedicate this report to my beloved parents, without their patience, support
understanding and most of all loves, the completion of work would have not been
possible.
iv
TABLE OF CONTENT
Abstract
i
Abstrak
ii
Dedication
iii
Acknowledgement
iv
Table of Content
v
List of Tables
ix
List of Figures
x
List Abbreviations
xiii
1. INTRODUCTION
1
1.1
Background
1
1.2
Problem Statement
2
1.3
Objectives
2
1.4
Scope
3
1.5
Expected Result
3
1.6
Content
3
2. LITERATURE REVIEW
5
2.1
Introduction
5
2.2
Vision System
5
2.3
Machine Vision Component
7
2.3.1
Vision Sensor Types
9
2.3.1.1
CCD
9
2.3.1.2
CMOS
14
2.3.1.3
Comparison between CCD and CMOS
17
2.3.2
Image Processing Technique
18
2.3.2.1
Smoothing Technique
20
2.3.2.1.1 Lowpass Filter
20
2.3.2.1.2 Local Average
21
2.3.2.1.3 Gaussian Filter
21
v
2.3.2.1.4 Median Filter
21
2.3.2.2
22
Edge Detection Technique
2.3.2.2.1 Prewitt Operator
23
2.3.2.2.2 Sobel Operator
24
2.3.2.2.3 Roberts Operator
24
2.4
Sorting System
25
2.4.1
Robot
26
2.4.2
Types of Robot Manipulator
27
2.5
Conveyor
29
2.5.1
Traditional Conveyor
30
2.5.2
Evolution of Conveyor
30
2.6
Measurement Devices
31
2.6.1
Analogue-to-Digital Converter
32
2.6.2
Photoelectric Sensor
33
2.7
Programmable Logic Controller (PLC)
36
2.8
Summary
38
3. METHODOLOGY
39
3.1
Introduction
39
3.2
Overall Research Strategies
39
3.3
Project Structure
43
3.3.1
Image Processing Procedures
45
3.4
Characterization of System Components
46
3.4.1
Conveyor
47
3.4.2
Rhino Robot Arm
48
3.4.3
Webcam
49
3.4.4
Personal Computer
49
3.4.5
Photoelectric Sensor
50
3.4.6
Programmable Logic Controller (PLC)
52
3.5
Summary
52
4. EXPERIMENTAL SETUP AND PROCEDURES
55
4.1
55
Introduction
vi
4.2
Experimental Setup
55
4.2.1
Hardware Component
55
4.2.2
Software Component
58
4.2.2.1
Logic of Electrical Control System
58
4.2.2.2
PC Access
60
4.2.2.3
Graphical User Interface (GUI)
62
4.2.2.4
Image Processing Configuration
63
4.2.2.5
Pick and Place Configuration
66
4.3
Experimental Setup Procedures
67
4.4
Summary
68
5. RESULT AND DISCUSSION
69
5.1
Introduction
69
5.2
Results
69
5.2.1
Ladder Diagram
70
5.2.2
Development of I/O Server
71
5.2.3
Graphical User Interface Design and Data Binding
72
5.2.4
Vision Inspection
74
5.2.5
Sorting
75
5.3
Discussion
75
5.3.1
Circuit Board for Relay
76
5.3.2
Matched Timing and Sequence between Robot Arm
77
and Image Acquisition
5.3.3
Numeric Indicators
77
5.4
Suggestion
78
5.4.1
Robot Feedback
78
5.4.2
Robot Efficiency
79
5.5
Summary
79
6. CONCLUSION
80
REFERENCES
82
vii
APPENDICES
85
A
Ladder Diagram
86
B
Rhino Robot Programming
90
C
Block Diagram of GUI
92
viii
LIST OF TABLES
2.1
Feature and Performance Comparison of CCD and
18
CMOS
2.2
Correction Factor for Direct Reflective Photoelectric
35
Sensor
3.1
Gantt Chart for FYP I
53
3.2
Gantt Chart for FYP II
54
4.1
Step for PLC Configuration
59
4.2
Configuration of PC Access
61
4.3
GUI Configuration of Labview
62
4.4
Image Processing Configuration on VBAI
64
4.5
Rhino Robot Configuration using RoboTalkWindow
66
5.1
Condition for Combination of Colour and Shape
70
ix
LIST OF FIGURES
2.1
Basic Functions of Machine Vision System
6
2.2
The Coordinate Convention of Pixels That Represent Digital
7
Images
8
2.3
The Basic Components of a Vision System
2.4
The Example of CCD Image Sensor
10
2.5(a)
CCD Pixel
12
2.5(b)
Architecture
12
2.6
Basic Principle of The Charge-Coupled Device
13
2.7
CMOS Sensor is Implemented in Low-End Web-Cam
14
2.8
CMOS Vision Sensor Floorplan
15
2.9
Schematic of The RGB Colour Cube
19
2.10
The Level of Histogram of Gray Scales
19
2.11
Calculating The Median Value of a Pixel Neighbourhood
22
2.12
A 3 × 3 Region Mask
23
2.13
Prewitt Horizontal and Vertical Operators
23
2.14
Sobel Convolution Kernels
24
2.15
The Computation Method of a Pixel
24
2.16
Roberts Cross Convolution Masks
25
2.17
The Closed-Loop Control System
27
2.18
Cartesian Robot
27
2.19
Cylindrical Robot
28
2.20
Spherical Robot
28
2.21
Articulated Robot
29
2.22
Traditional Conveyor
30
2.23
Steps in Analog-to-Digital Conversion of Continuous Analog
32
Signals from Process
2.24
Thru-Beam Photoelectric Sensor Detection
34
2.25
A Reflection with Reflector Photoelectric Sensor Detection
34
Method
2.26
A Direct Reflection Photoelectric Sensor Detection Method
x
35
2.27
Siemens Programmable Logic Controller, S7-200
36
2.28
Components of a PLC
37
3.1
Overall Work Flow Chart
41
3.2
The Flow Chart Illustrates The Data Collection and Data Study
42
3.3
The Flow Chart of Proposed System
44
3.4
Image Processing Flow Chart
46
3.5
The Schematic Diagram of Integrated System
47
3.6
A Belt Conveyor
47
3.7
Rhino Robot Arm
48
3.8
A Home-Used Webcam
49
3.9
A Personal Computer
50
3.10
Photoelectric Sensor
51
3.11
Siemens S7-200 PLC
52
4.1
CAD Drawing for Webcam Attachment on Conveyer
56
4.2
Actual Frame Mounting on Conveyor
56
4.3
Designed Logic Circuit Board
57
4.4
Overall Experimental Setup
68
5.1
Actual Sorting Result That is Performed by Rhino Robot
70
5.2
Designed Ladder Diagram
71
5.3
Develops and I/O Server Module using PC Access
72
5.4
GUI of The Integrated System
73
5.5
Select The Source Item in Data Binding
73
5.6
State Program for Vision Inspection
74
5.7
Inspection Step for State: Circle Red: x1
75
5.8
Relay Circuit Board Used to Triggered PLC and Motor of
76
Conveyor
5.9
Relay Circuit Board Used to Trigger Robot from PLC Input
77
Signal
5.10
Numeric Indicators for Accept and Reject Counting
xi
78
A1
Ladder Diagram
85
A2
Ladder Diagram
86
A3
Ladder Diagram
87
A4
Ladder Diagram
88
A5
Ladder Diagram
89
C1
Block Diagram for GUI
92
xii
LIST OF ABBREVIATIONS
ADC
-
Analogue-to-Digital Converter
APS
-
Active Pixels
CAPS
-
Complementary Active Pixels Sensors
CCD
-
Charged-Couple Device
CCTV
-
Close Circuit Television
CID
-
Charge-Injection Devices
CMOS
-
Complementary Metal-Oxide Semiconductor
DOF
-
Degree of Freedom
GUI
-
Graphical User Interface
I/O
-
Input/Output
LED
-
Light Emitter Diode
OLE
-
Object Linking and Embedding
OPC
-
OLE for Process Control
PC
-
Personal Computer
PCB
-
Printed Circuit Board
PIC
-
Person-in-charge
PLC
-
Programmable Logic Controller
PMOS
-
P-type metal-oxide-semiconductor
PPP
-
Prismatic, Prismatic, Prismatic
PPS
-
Passive Pixels
PVC
-
Polyvinyl Chloride
RGB
-
Red, Green, Blue
RPP
-
Rotary, Prismatic, Prismatic
ROI
-
Region of Interest
RRP
-
Rotary, Rotary, Prismatic
RRR
-
Rotary, Rotary, Rotary
USB
-
Universal Serial Bus
VBAI
-
Vision Builder Automated Inspection
VDC
-
Voltage Direct Current
VLSI
-
Very-large-scale integration
xiii
2-D
-
Two dimensions
3-D
-
Three dimensions
xiv
CHAPTER 1
INTRODUCTION
1.1
Background
The essential goals in the implementation of automation system in the industries are
to provide improvement in human operator variances and operational efficiency. It is
known in that many cases, product quality consistency can vary depending on the
characteristic of human operators who are on duty. Hence, automation system should
be provided to improve this variance. The problem associated with human-assisted
operation is characterized by cycle time, repeatability and material handling.
Although operational efficiency has been given a top priority, human operators are
still exposed to high probability of injuries, long-term similar working scope in fast
motion, applications naked eyes inspection-based system and insufficient break time.
In order to improve operational efficiency and to improve safety of human operators,
there is a need for efficient, high repeatability and cost effective human-assisted
operation equipment. Mechanized and automized solution that are properly sized
with suitable modularized structure can greatly improve the safety of personnel as
well as efficiency and quality of production. In addition, such efficient automation
system with high repeatability can greatly improve the operational efficiency,
product quality and product consistency when applied in combination with intelligent
instrumentation and advanced control.
The mechanized and automized operation system include robot arm, programmed
conveyor system, inspection system and host computer are necessitated to implement
in industries purposely to increase operational efficiency and product quality. A 6
degree of freedom (D.O.F) of robot arm which possess wide working area has
1
enlarge the approaching region and various orientation. Therefore, it is a wise
solution to be used to substitute the folklore operation system like human operator in
industries by robot arm. Besides, the evolution of conveyor from manual operate to
automated conveyor system has enhanced the product handling and increase the
cycle time as well. The implementation of vision system for inspection in production
line becomes a booster for inspection compare to old-fashion naked eye inspection.
The mechanized and automized systems are integrated with intelligent vision system
enable the whole production processes execute in fully automatic. Thus, the
operational efficiency is increase effectively and greatly decreases the variances of
product quality.
1.2
Problem Statement
Nowadays, human-operator based industries require longer time to inspect using
naked eye and sort the products into the correct slot or container. This action of
inspection and sorting consume more time and results in the operators unable to meet
the cycle time. Low repeatability is one the effects of long working time on similar,
repeating working scope at fast motion. Therefore, the operators may get tired and
bored resulting in bad inspection and sorting quality of the products. Due to the
limitations, an automated inspection system based on a vision system integrated into
a pick-and-place robot is required.
1.3
Objectives
Based on the problem statements stated above, several objectives are to be obtained
at the end of this project. The objectives of this project are:
i.
To design visual-based inspection system
ii.
To classify and sort products using vision inspection system
2
1.4
Scope
This project will focus on vision-based inspection and products sorting. The scopes
of this project are:
i.
Image acquisition using a webcam.
ii.
Configure vision inspection algorithm based on Vision Builder Automated
Inspection (VBAI).
iii.
Validate vision inspection system by interfacing a pick-and-place robot, a
webcam, a personal computer, a programmable logic controller (PLC), and a
conveyor.
iv.
Classify product in term of different colours and shapes.
The fabrication of conveyor, robot arm and development of software will not be
covered in this project.
1.5
Expected Result
This project is expected to improve the inspection process in terms of colour sorting
and shape orientation for packaging purpose. The product will be conveyed and
inspected for its colour and differentiated in term of shape using image processing
technique. A robot then removes the product and places it in a correct slot at right
orientation.
1.6
Content
This report is presented in several chapters that describe the development of
automatic quality control system using vision system. Chapter 2 discusses the
Literature Review that describes about the related information from journals and
books. Chapter 3 describes the Methodology of the proposed work and shows a flow
chart of process that explains the operational process supported with some statement.
3
Chapter 4 explains the Experimental Setup and Procedures of the installation and
integration of vision-based inspection system and sorting system. Chapter 5
illustrates on Results and Discussion about the results obtained from experiment. The
important findings will be presented in a comprehensive manner. A comparison
between the desired result and the actual result are performed in order to identify any
limitations to the system that is being proposed. In addition, the projected objectives
will be reviewed to ensure that goals met. Furthermore, suitable recommendation for
further improvement to the system will be included. Chapter 6 will conclude the
entire project with summarize the main findings and review the scopes covered.
4
CHAPTER 2
LITERATURE REVIEW
2.1
Introduction
This chapter discusses the literature review related to the proposed vision-based
inspection system. Review of previously developed systems in literature review are
discussed and illustrated. This chapter discusses mainly the vision system, sorting
system, conveyor and sensors that are relevant to the development of the proposed
system.
2.2
Vision System
Vision system can be described as image acquisition, followed by image processing
and interpretation. The processed image data is transferred by computer for some
other useful application. Vision system is classified into two dimensional (2-D) and
three dimensional (3-D). 2-D is most suitable for industrial applications that include
dimension measurement, gauging, check for the presence of components and etc.
(Groover, 2000). Vision systems are widely used in inspection section in industry.
The implementation of vision system increase the efficiency and accuracy of
inspection process because the high repeatability and accuracy of machine has
overcome the disadvantages of human-based operation system. Thus, sales of 2-D
vision system outnumber those of 3-D systems by more than ten to one. Figure 2.1
shows the basic functions of machine vision system.
5
Figure 2.1: Basic functions of machine vision system (Groover, 2000).
Image data acquisition is accomplished using a video camera where the image data is
transferred to digitizing system to store the data for further analysis. The camera is
focused on the region of interest (ROI), and an image data is obtained by dividing the
viewing area into a matrix pixel, arranged in pixel arrays. The intensity value of a
pixel is converted by analog-to-digital converter (ADC) to equivalent digital value
which is proportional to the light intensity. An image is represented in continuous
with respect to the x- and y- coordinates and in amplitude as well. In order to convert
it to digital form, an image is needed to digitizing the coordinate value (sampling)
and digitizing the amplitude value (quantization). In practice, the arrangement of
sensor used will determines the method of sampling to generate the image (Gonzalez
and Woods, 2002). The result of sampling and quantization of an image function, f(x,
y) now become discrete quantities resulting in a digital image with M rows and N
columns based on the matrix shown in equation 2.1. The coordinate convention of
pixels that represent digital images is shown in figure 2.2.
6