Real Time Wireless Robotic Arm System By Using Leap Motion Controller.
i
Real Time Wireless Robotic Arm System by Using Leap Motion
Controller
ONG TING YONG
This Report Is Submitted In Partial Fulfillment Of Requirements For
The Bachelor Degree of Electronic Engineering (Computer
Engineering)
Faculty of Electronic and Computer Engineering
University Technical Malaysia Malacca
ii
JUNE 2016
iii
iv
v
Dedicated to my beloved father and mother
vi
ACKNOWLEDGEMENT
First of all, I would like to express my greatest gratitude to Dr. Nurulfajar Bin Abd
Manap, my thesis advisor and also my project supervisor for his guidance and advice
throughout this project. He helped me out with his greatest knowledge in order to solve
some of the problem I faced for my final year project and encourage me to participate
some of the competition in order to gain more valuable experience. Next, I would like to
thanks to my panel examiners Dr. Lim Kim Chuan and Dr. Syafeeza Binti Ahmad Radzi
for spending their valuable time in order to listen my project presentation.
Besides, I would like to thanks to all the staff of the Department of Electronic and
Computer Engineering (FKEKK), UTeM for their guidance and valuable comments.
Nonetheless, I would like to thanks to my teammate supervised under Dr. Nurulfajar Bin
Abd Manap as well for sharing their knowledge and experiences throughout this project.
Last but not least, I would like to thanks to my family and my friends for giving me
support throughout the implementation of this project and I would like to apology for all
the inconvenience that I might had made during this project implementation.
Thanks.
vii
ABSTRACT
This report describe the design and the integration of a 5 Degree Of Freedom
(DOF) robotic arm into a wireless robotic arm system. Nowadays, the evolution of the
technology from time to time bring lots of changes to the human life. One of the
evolution until nowadays is the wireless system. A wireless system is a system that can
convey the information between two or more points and it is not physically connected by
any electrical conductors. The advantages to go wireless can increase the mobility of the
technology, increase the scalability, save cost and also save spaces. In this project, a 5
DOF robotic arm controlled by the Leap Motion controller in wireless system are
proposed. Generally, a robotic arm have the same function as a normal human arm. It
can be used to perform some high-precision task and also handle some hazardous
situation for us. However, most of the robotic arm are wire connected to the system
which limit the mobility of the robotic system and took up more spaces compared with
the wireless system. Hence, the approach to integrate the robotic arm into wireless
system are proposed. All the process to integrate the robotic arm into a wireless system
will be presented in this thesis. Besides, the method to integrate the robotic arm into a
wireless system will be discussed and demonstrated. Furthermore at the end of this
project, a prototype robotic arm in wireless system will be presented and the results for
this project will be recorded into this thesis.
viii
ABSTRAK
Laporan ini menerangkan reka bentuk dan integrasi ke atas robotik lengan yang
mempunyai 5 darjah kebebasan kepada sistem wayarles. Pada zaman sekarang, evolusi
teknologi dari semasa ke semasa telah membawa banyak perubahan kepada kehidupan
manusia. Salah satu evolusi adalah sistem wayarles. Sistem wayarles ini merupakan
salah satu sistem yang menyampaikan maklumat diantara dua atau lebih sistem dan ia
tidak disambungkan secara fizikal oleh konduktor elektrik. Evolusi teknologi dalam
sistem wayarles ini membawa manfaat seperti meningkatkan mobiliti, meningkatkan
produktiviti, menjimatkan kos dan juga menjimatkan ruang. Dalam projek ini, robotic
lengan yang mempunyai 5 darjah kebebasan, dikawal melalui Leap Motion controller
secara sistem wayarless akan dibentangkan. Secara umumnya, robotic lengan ini
mempunyai fungsi seperti lengan manusia biasa. Ia boleh digunakan untuk
melaksanakan tugas yang memerlukan ketepatan tinggi dan juga digunkan untuk
menangani keadaan yang berbahaya untuk kita. Walau bagaimanapun, kebanyakan
robotic lengan yang digunakan adalah dalam sistem berwayar. Ia menghadkan mobiliti
sistem robotik yang digunakan dan ia mengambil ruang yang lebih berbanding dengan
sistem wayarles. Oleh itu, cara untuk mengintegrasikan robotic lengan kepada sistem
wayarles dicadangkan. Proses mengintegrasikan robotic lengan kepada sistem wayarles
akan dibentangkan dalam tesis ini. Selain itu, kaedah- kaedah yang digunakan untuk
mengintegrasikan robotic lengan ini akan dibincangkan. Pada keakhiran projek ini, satu
prototaip robotic lengan dalam sistem wayarles akan dibentangkan dan hasilnya akan
direkodkan ke dalam tesis ini.
ix
TABLE OF CONTENT
CHAPTER
1
CONTENT
PAGE
TITLE
i
VERIFICATION FORM
ii
AUTHOR RECOGNITION
iii
SUPERVISOR RECOGNITION
iv
DEDICATION
v
ACKNOWLEDGEMEN
vi
ABSTRACT
vii
ABSTRAK
viii
TABLE OF CONTENT
ix
LIST OF TABLE
xii
LIST OF FIGURE
xiii
LIST OF ABBREVIATIONS
xvii
INTRODUCTION
1.1 Background
1
1.2 Problem Statement
3
1.3 Objectives
4
1.4 Scope of Project
4
1.5 Report Structure
5
x
2
LITERATURE REVIEW
2.1 Introduction
7
2.2 Robotic Arm
7
2.3 Leap Motion Controller
9
2.4 NRF24L01 Transceiver Module
11
2.5 Wireless System
12
2.6 Reviews on Related Works
14
2.6.1 Wireless Robotic Arm via Bluetooth Connection
14
2.6.2 Wireless Robotic Arm via Wi-Fi Connection
16
2.6.3 Wireless Robotic Arm via ZigBee Connection
16
2.6.4 Wireless Robotic Arm via Radio Frequency Connection
17
2.6.5 Comparison between Related Works
19
2.7 Summary
3
19
METHODOLOGY
3.1 Introduction
20
3.2 Project Overview
20
3.2.1 Description
20
3.2.2 Block Diagram
22
3.2.3 Project Implementation
23
3.3 Hardware Part
24
3.3.1 Components and Hardware
24
3.3.2 Integration Process
26
3.4 Software Part
29
3.4.1 Leap Motion
29
3.4.2 Node.js
31
3.4.3 Arduino IDE
32
3.4.4 Notepad++
34
3.5 Programming Part
35
3.6 Summary
36
xi
4
RESULTS AND DISCUSSION
4.1 Introduction
37
4.2 Input Data from Leap Motion Controller
37
4.3 From JavaScript to Arduino
51
4.4 From Arduino Transmitter to Arduino Receiver
60
4.5 Arduino and Circuitry for Both Transmitter and Receiver Module 62
5
4.6 From Arduino to Robotic Arm
65
4.7 Visualizer
73
4.8 Summary
74
CONCLUSION AND RECOMMENDATION
5.1 Conclusion
75
5.2 Recommendation
76
REFRENCES
78
xii
LIST OF TABLES
NO.
TITLE
PAGE
2.1
NRF24L01 transceiver pin and pin description
12
2.2
Comparison of related work
19
3.1
Connection for NRF24L01 RF module and Arduino Mega 2560 board
27
3.2
Pin connection between Arduino Uno and Arduino Mega 2560
28
3.3
Connection between Arduino Mega 2560 (receiver) and servo robotic arm
28
3.4
Details for EmBoss EB-500W power supply
29
xiii
LIST OF FIGURES
NO.
TITLE
PAGE
2.1
Robotic Arm with 6 Degree of Freedom (DOF)
9
2.2
Schematic view of the Leap Motion controller
10
2.3
Leap Motion coordinate system
10
2.4
NRF24L01 transceiver module
11
2.5
Robotic system to handle dangerous task
14
2.6
HC-05 Bluetooth module
15
2.7
ZigBee module
17
2.8
NRF24L01 module
18
3.1
Block diagram of the project
22
3.2
PCB board, female header pin and jumper wires
24
3.3
Leap Motion controller
24
3.4
Arduino Mega 2560 and Arduino Uno board
25
3.5
NRF24L01 transceiver module
25
3.6
The hardware for the robotic arm
26
3.7
Leap Motion Control Panel
30
3.8
Leap Motion Visualizer
30
3.9
Node.js command prompt
31
3.10
Standard Firmata code from Firmata library
33
3.11
Receiver code (left) and transmitter code (right)
33
3.12
Notepad++
34
3.13
Executing the JavaScript coding
34
xiv
4.1
Sample data obtained from the Leap Motion controller when the hand is in
center position
38
4.2
Sample data obtained when the hand is positioned towards the left end
39
4.3
Sample data acquired when the hand moved towards the right end
39
4.4
Sample data acquired when the hand is positioned towards the upper end
40
4.5
Sample data obtained when the hand moved towards the lower end
40
4.6
Sample data acquired when the hand is positioned towards the front end
41
4.7
Sample data obtained when the hand moved towards the back end
41
4.8
Leap Motion controller tracking data in x, y and z axis
42
4.9
X-axis rotational data obtained from Leap Motion Controller
43
4.10
Sample data obtained when the hand's position is in the center of Z rotation
43
4.11
Sample data acquired when the hand rotated upward according X-axis
44
4.12
Sample data obtained when the hand spin downward according X-axis
44
4.13
Z-rotational data obtained when the hand rotated rightward
45
4.14
Z-rotational data acquired when the hand spin leftward
45
4.15
Leap Motion controller tracking data in x rotation and z rotation
46
4.16
Sample data obtained for the middle finger and ring finger
47
4.17
Sample data obtained when the fingers in x-axis opened
48
4.18
Sample data obtained when the fingers opened according y-axis
48
4.19
Sample data obtained when the fingers in z-axis opened
49
4.20
Sample data for finger distance when the fingers are closed
50
4.21
Sample data obtained when the fingers are opened
50
4.22
Sample data obtained when the hand is in center position
52
4.23
Sample data obtained when the hand moved to the left end
53
4.24
Sample data acquired when the hand is in the right end
53
4.25
Sample data acquired when the hand moved to the upper end
54
4.26
Sample data obtained when the hand is in the lower end
54
4.27
Sample data obtained when the hand rotated to the right end
55
4.28
Sample data acquired when the hand rotated to the left end
55
4.29
Sample data obtained when the hand rotated to the lower end
56
4.30
Sample data acquired when the hand rotated to the upper end
56
xv
4.31
Sample data for fingers in normal position
57
4.32
Sample data obtained when the fingers are in closed position
57
4.33
Sample data acquired when the fingers are in opened position
58
4.34
Sample of data when the data are grouped as struct data
58
4.35
Sample data obtained when the hand moved to the left end and the palm of
the hand rotated toward the right end with fingers closed in Leap Motion
controller
4.36
59
Sample data acquired when the hand stand still in middle position and the
palm of the hand rotated toward the left end with fingers opened in Leap
Motion controller
4.37
Schematic diagram designed for both transmitter and receiver module in
Arduino board
4.38
63
Transmitter board (Arduino Mega 2560+ Arduino Uno + NRF24L01
transceiver module) connected with host PC
4.41
62
Printed PCB layout for both transmitter and receiver module in Arduino
board
4.40
62
PCB layout routing for both transmitter and receiver module in Arduino
board
4.39
59
64
Receiver board (Arduino Mega 2560 + NRF24L01 transceiver module)
connected with robotic arm
64
4.42
Orange (Signal pin), Red (VCC pin), Black (ground pin)
65
4.43
Overall connection for the wireless robotic arm system
67
4.44
The robotic arm moved rightward according to the hand's position detected
in Leap Motion controller
4.45
The robotic arm moved leftward according to the hand's position detected in
Leap Motion controller
4.46
68
The robotic arm moved upward according to the hand's position detected in
Leap Motion controller
4.47
68
69
The robotic arm moved downward according to the hand's position detected
in Leap Motion controller
69
xvi
4.48
The robotic palm rotated leftward according to the palm's position detected
in Leap Motion controller
4.49
The robotic palm rotated rightward according to the palm's position detected
in Leap Motion controller
4.50
71
The robotic gripper opened according to the fingers detected in Leap Motion
controller
4.53
71
The robotic palm moved downward according to the hand's position detected
in Leap Motion controller
4.52
70
The robotic palm moved upward according to the palm's position detected in
Leap Motion controller
4.51
70
72
The robotic gripper closed according to the fingers detected in Leap Motion
controller
72
4.54
Visualizer by Leap Motion App Home software
73
4.55
Visualizer using HTML and JavaScript coding
73
xvii
LIST OF ABBREVIATIONS
COM
-
Communication port
DOF
-
Degree of Freedom
FIFO
-
First In First Out
GFSK
-
Gaussian Frequency Shift Keying
IDE
-
Integrated Development Environment
ISM
-
Industrial, Scientific and Medical
PC
-
Personal Computer
PCB
-
Printed Circuit Board
PWM
-
Pulse Width Modulation
RF
-
Radio Frequency
RX
-
Receiver
SPI
-
Serial Peripheral Interface
TX
-
Transmitter
USB
-
Universal Serial Bus
Wi-Fi
-
Wireless Fidelity
1
CHAPTER 1
INTRODUCTION
In this chapter, all the project introduction, background, problem statement,
objectives, scope of project and report structure will be discussed. This project is all
about controlling a real time robotic arm in wireless system by using the NRF24L01
transceiver module and Leap Motion controller. The robotic arm moved according to the
hand detected by the Leap Motion controller while NRF24L01 transceiver module act as
the transmitter and receiver module for the robotic arm system. The background of this
project will be discussed next, continue with problem statement, objectives, scope of
project and the structure of this report.
1.1 Background
Nowadays, the invention of the robotic system grows rapidly from time to time
to ease human in daily activities and serve the world for a better future. In order to build
out a robotic system, the three main branch to develop a good robotic system are
mechanical engineering, electronic engineering and computer science engineering. The
mechanical engineering will handle the hardware part, the computer science engineering
will handle the software part while electronic engineering will handle both hardware and
2
software part. The hardware developer will cover up the design and build up the robotic
system while the software developer will focus on how to function the robotic system.
Different robotic system are designed according to the requirement of the user to
perform specific tasks. The evolution of the robotic system serve human in wide range
of areas such as performing hazardous task for human, environmental interaction and
navigation, resemble human in appearance, education, trainings and much more.
Generally, most of the robotic system are used as industrial robots to perform
human task. It can be used to aid human in performing specific task which could bring
long term or short term harm for humans, aid in hazardous task, dirty task or even boring
task for humans. Besides, robots are widely used in many fields such as underwater
exploration, space exploration, military services, fight against crime, car production,
duct cleaning, fix oil spills, investigate hazardous environment and so on. To design and
construct a robotic system, the most important element to power up a robotic system is
the power source. The power source can be batteries, solar power, pneumatic, nuclear
energy, hydroelectricity and so on. Without the power sources, it is impossible to
function the robotic system. Next, the second element to build up a robotic system is the
actuator. Actuator is the control system for the robotic system and it act as a motor to
rotate and move the robotic system accordingly. Normally, the actuators are powered up
by using the power sources and rotate according to the input signal from the system.
Next, the third element to function the robotic system is the computer programming
code. Programming code are the most important element for the robotic system to move.
Depend on what robot designed, programming code are the essential element to program
a robot on how the robot move according to the input signal. It can be C programming,
C# programming, VB programming, HTML programming, JavaScript programming and
so on.
In the past few years, the most common controller for a robotic system are
keypad, mouse, keyboard or joystick. However, with the growth of the technologies,
numerous of the controller has been invented especially in human-computer interaction
technologies. Besides, most of the robotic system are interconnected via wired
3
connection. With the latest technologies nowadays, the interaction between the human
and the robotic system in wired system can be done as well by using wireless system.
With the integration of wireless communication over the wired communication, the
advantages are greatly increase in the mobility for the robotic system, no cables to mess
up the system, save cost and controllable from far to handle tasks.
The main purpose of this project is to build a wireless robotic arm system
controlled by using a hand gestures controller to control the robotic arm simultaneously.
This robotic arm system will perform according to our hand gesture movement and the
connection between the controller and the robotic arm will be connected via wireless
connection.
1.2 Problem Statement
Although there are various methods to control a robotic arm system, not all
controller are as simplest as possible to control a robotic arm system. Some common
controller to control a robotic arm system include mouse controller, keyboard controller,
joystick controller, keypad controller and so on. Although all these controller are the
most common controller to control a robotic arm system, there are still some difficulties
to use these controller. The difficulties to use these controller include lack of the
understanding about functionality of each button, time consuming to think and move the
robotic arm according ideal movement and hard to control the robotic arm precisely and
accurately. In order to overcome these difficulties, a better solution to control the robotic
arm can be implemented by using the Leap Motion controller. Leap Motion controller, a
hand gestures sensor controller is a new technology that tracks the user's hand
simultaneously with accuracy approximately 0.01mm and a frame rate up to 300 frame
per second. It can detect the user's hand within its surface area of 24 cm² and field of
view up to 150°. It is the best solution to control the robotic arm since it is user friendly,
easy to understand, easy to use and no trainings are required.
4
Besides, the robotic arm nowadays are fixed in a workplace and connected with
the controller via wired connection. If the robotic arm are fixed in a place and connected
to the controller via cable connection, it limit the mobility of the robotic arm to move
from place to place and control it from far distance. Furthermore, exposed cable without
cares could easily damage by the rats, cleaners or even the workers. It may cause power
failure, system failure or even worst is sending someone to the emergency room. With
the integration of the robotic arm into a wireless system, it allow the users to control the
robotic arm from far or handle some dangerous activities such as handling toxic
substances, bomb disposal or exploring hazardous environments. Besides, it can be used
for asepsis preservation in operating room, military service, deep sea discovery or space
discovery to discover the existing and educate the future about the discovery.
1.3 Objectives
The purpose of this project is to build up a robotic arm in wireless system by using the
Leap Motion controller to detect our hand gesture movement and move the robotic arm
simultaneously. In short, the objectives are:
1.3.1 To assemble the robotic arm with Leap Motion controller in wired system.
1.3.2 To integrate the robotic arm system into a wireless robotic arm system.
1.3.3 To develop a real time wireless robotic arm system.
1.4 Scope of Project
The scope of work in this project are divided into three main parts where the first
part of this project will focus on how to get the Leap Motion data from the Leap Motion
controller while the second part of this project will focus on how to utilize the Leap
Motion data to control a 5 DOF robotic arm in wired connection. If the robotic arm in
the second part moved simultaneously according to the hand detected by the Leap
Motion controller, the third part of this project will be the integration of this robotic arm
system into a wireless robotic arm system.
5
For Leap Motion controller, the Leap Motion controller is a hand gesture sensor
in built with two monochromatic IR cameras and three infrared LEDs to interpret the
user's hand accurately and precisely. The Leap Motion data can be obtained via some
programming language and perform some formula calculation to convert it into PWM
signal from 0 to 255 corresponds to the angular range of 0º to 180º [16] for the servo
motors. Next, the PWM signal are feed into a microcontroller to control each servo
motor in the robotic arm simultaneously. When the robotic arm move according to the
hand detected by the Leap Motion controller, NRF24L01 transceiver module can be
added to integrate the robotic arm into a wireless robotic arm system. One will be added
to the microcontroller attached with the robotic arm while another one will be added to
another microcontroller connected with the PC. NRF24L01 transceiver module were
chosen for this project because this module are the cheapest module with high data rate,
low power consumption and support up to 100 meter for wireless communication.
1.5 Report Structure
For the structure of this report, this report consists of five different chapters. The
first chapter of this report will be the introduction of this project and the second chapter
will be the literature review of this project. The methodology used for this project falls
under Chapter 3 while result and discussion of this project will be included into Chapter
4. Lastly, the last part which is the conclusion and the recommendation of this project
will be described in Chapter 5.
In Chapter 1, this chapter will introduce about the introduction of this project.
The background of this project and the problem statement to be solved in this project
were stated. Besides, the objectives of this project and the scope of this project were
mentioned in this chapter. For Chapter 2, this chapter will fully discuss about the
literature reviews related to this project. The literature review gathered from various
sources are divided into two main parts, the first part will be the basic theory and the
principles for this project while the second part will be the overall review from some
journals related to this project.
6
For Chapter 3, this chapter discuss about the methodology used for this project.
In order to achieve the objectives of this project, some problem solving methods and
work are stated here. Moreover, the process to construct and the implementation for the
wireless robotic arm system will be further discussed in this chapter. For Chapter 4, this
chapter identifies the results obtained from this project. Besides, this chapter will present
and explain about the project setup and some sort of problem encountered throughout
the project. Lastly for Chapter 5, this chapter will be the conclusion for this project.
Some future work and recommendations are stated here in order to improve this project.
7
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
This chapter discusses about some published information regarding this project
and it will cover with some basic theories together with its principles for a better
understanding about this project. Besides, there will be a section discusses about some
published project related to this project and a short comparison between each project.
Furthermore, this part will conclude the entire summary for the literature review and the
alternative to be used throughout this project.
2.2 Robotic Arm
Generally, robotic arm are similar as the human arm and it is programmable
according to the requirement of the users. It is not easy to build up a robotic arm system
because it required the branch of electronic engineering to identify the chips or
component used in the robotic arm, mechanical engineering to design and to construct
the robotic arm, electrical engineering to control the power input and the power output
for each servo motor and computer science to program the operation of the robotic arm
system. Generally, a robotic arm system are classified into two main categories. The first
Real Time Wireless Robotic Arm System by Using Leap Motion
Controller
ONG TING YONG
This Report Is Submitted In Partial Fulfillment Of Requirements For
The Bachelor Degree of Electronic Engineering (Computer
Engineering)
Faculty of Electronic and Computer Engineering
University Technical Malaysia Malacca
ii
JUNE 2016
iii
iv
v
Dedicated to my beloved father and mother
vi
ACKNOWLEDGEMENT
First of all, I would like to express my greatest gratitude to Dr. Nurulfajar Bin Abd
Manap, my thesis advisor and also my project supervisor for his guidance and advice
throughout this project. He helped me out with his greatest knowledge in order to solve
some of the problem I faced for my final year project and encourage me to participate
some of the competition in order to gain more valuable experience. Next, I would like to
thanks to my panel examiners Dr. Lim Kim Chuan and Dr. Syafeeza Binti Ahmad Radzi
for spending their valuable time in order to listen my project presentation.
Besides, I would like to thanks to all the staff of the Department of Electronic and
Computer Engineering (FKEKK), UTeM for their guidance and valuable comments.
Nonetheless, I would like to thanks to my teammate supervised under Dr. Nurulfajar Bin
Abd Manap as well for sharing their knowledge and experiences throughout this project.
Last but not least, I would like to thanks to my family and my friends for giving me
support throughout the implementation of this project and I would like to apology for all
the inconvenience that I might had made during this project implementation.
Thanks.
vii
ABSTRACT
This report describe the design and the integration of a 5 Degree Of Freedom
(DOF) robotic arm into a wireless robotic arm system. Nowadays, the evolution of the
technology from time to time bring lots of changes to the human life. One of the
evolution until nowadays is the wireless system. A wireless system is a system that can
convey the information between two or more points and it is not physically connected by
any electrical conductors. The advantages to go wireless can increase the mobility of the
technology, increase the scalability, save cost and also save spaces. In this project, a 5
DOF robotic arm controlled by the Leap Motion controller in wireless system are
proposed. Generally, a robotic arm have the same function as a normal human arm. It
can be used to perform some high-precision task and also handle some hazardous
situation for us. However, most of the robotic arm are wire connected to the system
which limit the mobility of the robotic system and took up more spaces compared with
the wireless system. Hence, the approach to integrate the robotic arm into wireless
system are proposed. All the process to integrate the robotic arm into a wireless system
will be presented in this thesis. Besides, the method to integrate the robotic arm into a
wireless system will be discussed and demonstrated. Furthermore at the end of this
project, a prototype robotic arm in wireless system will be presented and the results for
this project will be recorded into this thesis.
viii
ABSTRAK
Laporan ini menerangkan reka bentuk dan integrasi ke atas robotik lengan yang
mempunyai 5 darjah kebebasan kepada sistem wayarles. Pada zaman sekarang, evolusi
teknologi dari semasa ke semasa telah membawa banyak perubahan kepada kehidupan
manusia. Salah satu evolusi adalah sistem wayarles. Sistem wayarles ini merupakan
salah satu sistem yang menyampaikan maklumat diantara dua atau lebih sistem dan ia
tidak disambungkan secara fizikal oleh konduktor elektrik. Evolusi teknologi dalam
sistem wayarles ini membawa manfaat seperti meningkatkan mobiliti, meningkatkan
produktiviti, menjimatkan kos dan juga menjimatkan ruang. Dalam projek ini, robotic
lengan yang mempunyai 5 darjah kebebasan, dikawal melalui Leap Motion controller
secara sistem wayarless akan dibentangkan. Secara umumnya, robotic lengan ini
mempunyai fungsi seperti lengan manusia biasa. Ia boleh digunakan untuk
melaksanakan tugas yang memerlukan ketepatan tinggi dan juga digunkan untuk
menangani keadaan yang berbahaya untuk kita. Walau bagaimanapun, kebanyakan
robotic lengan yang digunakan adalah dalam sistem berwayar. Ia menghadkan mobiliti
sistem robotik yang digunakan dan ia mengambil ruang yang lebih berbanding dengan
sistem wayarles. Oleh itu, cara untuk mengintegrasikan robotic lengan kepada sistem
wayarles dicadangkan. Proses mengintegrasikan robotic lengan kepada sistem wayarles
akan dibentangkan dalam tesis ini. Selain itu, kaedah- kaedah yang digunakan untuk
mengintegrasikan robotic lengan ini akan dibincangkan. Pada keakhiran projek ini, satu
prototaip robotic lengan dalam sistem wayarles akan dibentangkan dan hasilnya akan
direkodkan ke dalam tesis ini.
ix
TABLE OF CONTENT
CHAPTER
1
CONTENT
PAGE
TITLE
i
VERIFICATION FORM
ii
AUTHOR RECOGNITION
iii
SUPERVISOR RECOGNITION
iv
DEDICATION
v
ACKNOWLEDGEMEN
vi
ABSTRACT
vii
ABSTRAK
viii
TABLE OF CONTENT
ix
LIST OF TABLE
xii
LIST OF FIGURE
xiii
LIST OF ABBREVIATIONS
xvii
INTRODUCTION
1.1 Background
1
1.2 Problem Statement
3
1.3 Objectives
4
1.4 Scope of Project
4
1.5 Report Structure
5
x
2
LITERATURE REVIEW
2.1 Introduction
7
2.2 Robotic Arm
7
2.3 Leap Motion Controller
9
2.4 NRF24L01 Transceiver Module
11
2.5 Wireless System
12
2.6 Reviews on Related Works
14
2.6.1 Wireless Robotic Arm via Bluetooth Connection
14
2.6.2 Wireless Robotic Arm via Wi-Fi Connection
16
2.6.3 Wireless Robotic Arm via ZigBee Connection
16
2.6.4 Wireless Robotic Arm via Radio Frequency Connection
17
2.6.5 Comparison between Related Works
19
2.7 Summary
3
19
METHODOLOGY
3.1 Introduction
20
3.2 Project Overview
20
3.2.1 Description
20
3.2.2 Block Diagram
22
3.2.3 Project Implementation
23
3.3 Hardware Part
24
3.3.1 Components and Hardware
24
3.3.2 Integration Process
26
3.4 Software Part
29
3.4.1 Leap Motion
29
3.4.2 Node.js
31
3.4.3 Arduino IDE
32
3.4.4 Notepad++
34
3.5 Programming Part
35
3.6 Summary
36
xi
4
RESULTS AND DISCUSSION
4.1 Introduction
37
4.2 Input Data from Leap Motion Controller
37
4.3 From JavaScript to Arduino
51
4.4 From Arduino Transmitter to Arduino Receiver
60
4.5 Arduino and Circuitry for Both Transmitter and Receiver Module 62
5
4.6 From Arduino to Robotic Arm
65
4.7 Visualizer
73
4.8 Summary
74
CONCLUSION AND RECOMMENDATION
5.1 Conclusion
75
5.2 Recommendation
76
REFRENCES
78
xii
LIST OF TABLES
NO.
TITLE
PAGE
2.1
NRF24L01 transceiver pin and pin description
12
2.2
Comparison of related work
19
3.1
Connection for NRF24L01 RF module and Arduino Mega 2560 board
27
3.2
Pin connection between Arduino Uno and Arduino Mega 2560
28
3.3
Connection between Arduino Mega 2560 (receiver) and servo robotic arm
28
3.4
Details for EmBoss EB-500W power supply
29
xiii
LIST OF FIGURES
NO.
TITLE
PAGE
2.1
Robotic Arm with 6 Degree of Freedom (DOF)
9
2.2
Schematic view of the Leap Motion controller
10
2.3
Leap Motion coordinate system
10
2.4
NRF24L01 transceiver module
11
2.5
Robotic system to handle dangerous task
14
2.6
HC-05 Bluetooth module
15
2.7
ZigBee module
17
2.8
NRF24L01 module
18
3.1
Block diagram of the project
22
3.2
PCB board, female header pin and jumper wires
24
3.3
Leap Motion controller
24
3.4
Arduino Mega 2560 and Arduino Uno board
25
3.5
NRF24L01 transceiver module
25
3.6
The hardware for the robotic arm
26
3.7
Leap Motion Control Panel
30
3.8
Leap Motion Visualizer
30
3.9
Node.js command prompt
31
3.10
Standard Firmata code from Firmata library
33
3.11
Receiver code (left) and transmitter code (right)
33
3.12
Notepad++
34
3.13
Executing the JavaScript coding
34
xiv
4.1
Sample data obtained from the Leap Motion controller when the hand is in
center position
38
4.2
Sample data obtained when the hand is positioned towards the left end
39
4.3
Sample data acquired when the hand moved towards the right end
39
4.4
Sample data acquired when the hand is positioned towards the upper end
40
4.5
Sample data obtained when the hand moved towards the lower end
40
4.6
Sample data acquired when the hand is positioned towards the front end
41
4.7
Sample data obtained when the hand moved towards the back end
41
4.8
Leap Motion controller tracking data in x, y and z axis
42
4.9
X-axis rotational data obtained from Leap Motion Controller
43
4.10
Sample data obtained when the hand's position is in the center of Z rotation
43
4.11
Sample data acquired when the hand rotated upward according X-axis
44
4.12
Sample data obtained when the hand spin downward according X-axis
44
4.13
Z-rotational data obtained when the hand rotated rightward
45
4.14
Z-rotational data acquired when the hand spin leftward
45
4.15
Leap Motion controller tracking data in x rotation and z rotation
46
4.16
Sample data obtained for the middle finger and ring finger
47
4.17
Sample data obtained when the fingers in x-axis opened
48
4.18
Sample data obtained when the fingers opened according y-axis
48
4.19
Sample data obtained when the fingers in z-axis opened
49
4.20
Sample data for finger distance when the fingers are closed
50
4.21
Sample data obtained when the fingers are opened
50
4.22
Sample data obtained when the hand is in center position
52
4.23
Sample data obtained when the hand moved to the left end
53
4.24
Sample data acquired when the hand is in the right end
53
4.25
Sample data acquired when the hand moved to the upper end
54
4.26
Sample data obtained when the hand is in the lower end
54
4.27
Sample data obtained when the hand rotated to the right end
55
4.28
Sample data acquired when the hand rotated to the left end
55
4.29
Sample data obtained when the hand rotated to the lower end
56
4.30
Sample data acquired when the hand rotated to the upper end
56
xv
4.31
Sample data for fingers in normal position
57
4.32
Sample data obtained when the fingers are in closed position
57
4.33
Sample data acquired when the fingers are in opened position
58
4.34
Sample of data when the data are grouped as struct data
58
4.35
Sample data obtained when the hand moved to the left end and the palm of
the hand rotated toward the right end with fingers closed in Leap Motion
controller
4.36
59
Sample data acquired when the hand stand still in middle position and the
palm of the hand rotated toward the left end with fingers opened in Leap
Motion controller
4.37
Schematic diagram designed for both transmitter and receiver module in
Arduino board
4.38
63
Transmitter board (Arduino Mega 2560+ Arduino Uno + NRF24L01
transceiver module) connected with host PC
4.41
62
Printed PCB layout for both transmitter and receiver module in Arduino
board
4.40
62
PCB layout routing for both transmitter and receiver module in Arduino
board
4.39
59
64
Receiver board (Arduino Mega 2560 + NRF24L01 transceiver module)
connected with robotic arm
64
4.42
Orange (Signal pin), Red (VCC pin), Black (ground pin)
65
4.43
Overall connection for the wireless robotic arm system
67
4.44
The robotic arm moved rightward according to the hand's position detected
in Leap Motion controller
4.45
The robotic arm moved leftward according to the hand's position detected in
Leap Motion controller
4.46
68
The robotic arm moved upward according to the hand's position detected in
Leap Motion controller
4.47
68
69
The robotic arm moved downward according to the hand's position detected
in Leap Motion controller
69
xvi
4.48
The robotic palm rotated leftward according to the palm's position detected
in Leap Motion controller
4.49
The robotic palm rotated rightward according to the palm's position detected
in Leap Motion controller
4.50
71
The robotic gripper opened according to the fingers detected in Leap Motion
controller
4.53
71
The robotic palm moved downward according to the hand's position detected
in Leap Motion controller
4.52
70
The robotic palm moved upward according to the palm's position detected in
Leap Motion controller
4.51
70
72
The robotic gripper closed according to the fingers detected in Leap Motion
controller
72
4.54
Visualizer by Leap Motion App Home software
73
4.55
Visualizer using HTML and JavaScript coding
73
xvii
LIST OF ABBREVIATIONS
COM
-
Communication port
DOF
-
Degree of Freedom
FIFO
-
First In First Out
GFSK
-
Gaussian Frequency Shift Keying
IDE
-
Integrated Development Environment
ISM
-
Industrial, Scientific and Medical
PC
-
Personal Computer
PCB
-
Printed Circuit Board
PWM
-
Pulse Width Modulation
RF
-
Radio Frequency
RX
-
Receiver
SPI
-
Serial Peripheral Interface
TX
-
Transmitter
USB
-
Universal Serial Bus
Wi-Fi
-
Wireless Fidelity
1
CHAPTER 1
INTRODUCTION
In this chapter, all the project introduction, background, problem statement,
objectives, scope of project and report structure will be discussed. This project is all
about controlling a real time robotic arm in wireless system by using the NRF24L01
transceiver module and Leap Motion controller. The robotic arm moved according to the
hand detected by the Leap Motion controller while NRF24L01 transceiver module act as
the transmitter and receiver module for the robotic arm system. The background of this
project will be discussed next, continue with problem statement, objectives, scope of
project and the structure of this report.
1.1 Background
Nowadays, the invention of the robotic system grows rapidly from time to time
to ease human in daily activities and serve the world for a better future. In order to build
out a robotic system, the three main branch to develop a good robotic system are
mechanical engineering, electronic engineering and computer science engineering. The
mechanical engineering will handle the hardware part, the computer science engineering
will handle the software part while electronic engineering will handle both hardware and
2
software part. The hardware developer will cover up the design and build up the robotic
system while the software developer will focus on how to function the robotic system.
Different robotic system are designed according to the requirement of the user to
perform specific tasks. The evolution of the robotic system serve human in wide range
of areas such as performing hazardous task for human, environmental interaction and
navigation, resemble human in appearance, education, trainings and much more.
Generally, most of the robotic system are used as industrial robots to perform
human task. It can be used to aid human in performing specific task which could bring
long term or short term harm for humans, aid in hazardous task, dirty task or even boring
task for humans. Besides, robots are widely used in many fields such as underwater
exploration, space exploration, military services, fight against crime, car production,
duct cleaning, fix oil spills, investigate hazardous environment and so on. To design and
construct a robotic system, the most important element to power up a robotic system is
the power source. The power source can be batteries, solar power, pneumatic, nuclear
energy, hydroelectricity and so on. Without the power sources, it is impossible to
function the robotic system. Next, the second element to build up a robotic system is the
actuator. Actuator is the control system for the robotic system and it act as a motor to
rotate and move the robotic system accordingly. Normally, the actuators are powered up
by using the power sources and rotate according to the input signal from the system.
Next, the third element to function the robotic system is the computer programming
code. Programming code are the most important element for the robotic system to move.
Depend on what robot designed, programming code are the essential element to program
a robot on how the robot move according to the input signal. It can be C programming,
C# programming, VB programming, HTML programming, JavaScript programming and
so on.
In the past few years, the most common controller for a robotic system are
keypad, mouse, keyboard or joystick. However, with the growth of the technologies,
numerous of the controller has been invented especially in human-computer interaction
technologies. Besides, most of the robotic system are interconnected via wired
3
connection. With the latest technologies nowadays, the interaction between the human
and the robotic system in wired system can be done as well by using wireless system.
With the integration of wireless communication over the wired communication, the
advantages are greatly increase in the mobility for the robotic system, no cables to mess
up the system, save cost and controllable from far to handle tasks.
The main purpose of this project is to build a wireless robotic arm system
controlled by using a hand gestures controller to control the robotic arm simultaneously.
This robotic arm system will perform according to our hand gesture movement and the
connection between the controller and the robotic arm will be connected via wireless
connection.
1.2 Problem Statement
Although there are various methods to control a robotic arm system, not all
controller are as simplest as possible to control a robotic arm system. Some common
controller to control a robotic arm system include mouse controller, keyboard controller,
joystick controller, keypad controller and so on. Although all these controller are the
most common controller to control a robotic arm system, there are still some difficulties
to use these controller. The difficulties to use these controller include lack of the
understanding about functionality of each button, time consuming to think and move the
robotic arm according ideal movement and hard to control the robotic arm precisely and
accurately. In order to overcome these difficulties, a better solution to control the robotic
arm can be implemented by using the Leap Motion controller. Leap Motion controller, a
hand gestures sensor controller is a new technology that tracks the user's hand
simultaneously with accuracy approximately 0.01mm and a frame rate up to 300 frame
per second. It can detect the user's hand within its surface area of 24 cm² and field of
view up to 150°. It is the best solution to control the robotic arm since it is user friendly,
easy to understand, easy to use and no trainings are required.
4
Besides, the robotic arm nowadays are fixed in a workplace and connected with
the controller via wired connection. If the robotic arm are fixed in a place and connected
to the controller via cable connection, it limit the mobility of the robotic arm to move
from place to place and control it from far distance. Furthermore, exposed cable without
cares could easily damage by the rats, cleaners or even the workers. It may cause power
failure, system failure or even worst is sending someone to the emergency room. With
the integration of the robotic arm into a wireless system, it allow the users to control the
robotic arm from far or handle some dangerous activities such as handling toxic
substances, bomb disposal or exploring hazardous environments. Besides, it can be used
for asepsis preservation in operating room, military service, deep sea discovery or space
discovery to discover the existing and educate the future about the discovery.
1.3 Objectives
The purpose of this project is to build up a robotic arm in wireless system by using the
Leap Motion controller to detect our hand gesture movement and move the robotic arm
simultaneously. In short, the objectives are:
1.3.1 To assemble the robotic arm with Leap Motion controller in wired system.
1.3.2 To integrate the robotic arm system into a wireless robotic arm system.
1.3.3 To develop a real time wireless robotic arm system.
1.4 Scope of Project
The scope of work in this project are divided into three main parts where the first
part of this project will focus on how to get the Leap Motion data from the Leap Motion
controller while the second part of this project will focus on how to utilize the Leap
Motion data to control a 5 DOF robotic arm in wired connection. If the robotic arm in
the second part moved simultaneously according to the hand detected by the Leap
Motion controller, the third part of this project will be the integration of this robotic arm
system into a wireless robotic arm system.
5
For Leap Motion controller, the Leap Motion controller is a hand gesture sensor
in built with two monochromatic IR cameras and three infrared LEDs to interpret the
user's hand accurately and precisely. The Leap Motion data can be obtained via some
programming language and perform some formula calculation to convert it into PWM
signal from 0 to 255 corresponds to the angular range of 0º to 180º [16] for the servo
motors. Next, the PWM signal are feed into a microcontroller to control each servo
motor in the robotic arm simultaneously. When the robotic arm move according to the
hand detected by the Leap Motion controller, NRF24L01 transceiver module can be
added to integrate the robotic arm into a wireless robotic arm system. One will be added
to the microcontroller attached with the robotic arm while another one will be added to
another microcontroller connected with the PC. NRF24L01 transceiver module were
chosen for this project because this module are the cheapest module with high data rate,
low power consumption and support up to 100 meter for wireless communication.
1.5 Report Structure
For the structure of this report, this report consists of five different chapters. The
first chapter of this report will be the introduction of this project and the second chapter
will be the literature review of this project. The methodology used for this project falls
under Chapter 3 while result and discussion of this project will be included into Chapter
4. Lastly, the last part which is the conclusion and the recommendation of this project
will be described in Chapter 5.
In Chapter 1, this chapter will introduce about the introduction of this project.
The background of this project and the problem statement to be solved in this project
were stated. Besides, the objectives of this project and the scope of this project were
mentioned in this chapter. For Chapter 2, this chapter will fully discuss about the
literature reviews related to this project. The literature review gathered from various
sources are divided into two main parts, the first part will be the basic theory and the
principles for this project while the second part will be the overall review from some
journals related to this project.
6
For Chapter 3, this chapter discuss about the methodology used for this project.
In order to achieve the objectives of this project, some problem solving methods and
work are stated here. Moreover, the process to construct and the implementation for the
wireless robotic arm system will be further discussed in this chapter. For Chapter 4, this
chapter identifies the results obtained from this project. Besides, this chapter will present
and explain about the project setup and some sort of problem encountered throughout
the project. Lastly for Chapter 5, this chapter will be the conclusion for this project.
Some future work and recommendations are stated here in order to improve this project.
7
CHAPTER 2
LITERATURE REVIEW
2.1 Introduction
This chapter discusses about some published information regarding this project
and it will cover with some basic theories together with its principles for a better
understanding about this project. Besides, there will be a section discusses about some
published project related to this project and a short comparison between each project.
Furthermore, this part will conclude the entire summary for the literature review and the
alternative to be used throughout this project.
2.2 Robotic Arm
Generally, robotic arm are similar as the human arm and it is programmable
according to the requirement of the users. It is not easy to build up a robotic arm system
because it required the branch of electronic engineering to identify the chips or
component used in the robotic arm, mechanical engineering to design and to construct
the robotic arm, electrical engineering to control the power input and the power output
for each servo motor and computer science to program the operation of the robotic arm
system. Generally, a robotic arm system are classified into two main categories. The first