Speech Recognition Over Electrical/Electronic Appliances.
SPEECH RECOGNITION OVER ELECTRICAL/ELECTRONIC APPLIANCES
LEE KAM WAH
This report is submitted in partial fulfillment of the requirements for the award of Bachelor of Electronic Engineering (Computer Engineering) With Honors
Faculty of Electronic and Computer Engineering Universiti Teknikal Malaysia Melaka
(2)
ii
UNIVERSTI TEKNIKAL MALAYSIA MELAKA
FAKULTI KEJURUTERAAN ELEKTRONIK DAN KEJURUTERAAN KOMPUTER BORANG PENGESAHAN STATUS LAPORAN
PROJEK SARJANA MUDA II
Tajuk Projek : Speech Recognition Over Electrical/Electronic Appliances
Sesi
Pengajian : 2009/2010
Saya LEE KAM WAH
mengaku membenarkan Laporan Projek Sarjana Muda ini disimpan di Perpustakaan dengan syarat-syarat kegunaan seperti berikut:
1. Laporan adalah hakmilik Universiti Teknikal Malaysia Melaka.
2. Perpustakaan dibenarkan membuat salinan untuk tujuan pengajian sahaja.
3. Perpustakaan dibenarkan membuat salinan laporan ini sebagai bahan pertukaran antara institusi pengajian tinggi.
4. Sila tandakan ( √ ) :
SULIT*
(Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972)
TERHAD* (Mengandungi maklumat terhad yang telah ditentukan oleh organisasi/badan di mana penyelidikan dijalankan)
TIDAK TERHAD
Disahkan oleh:
__________________________ ___________________________________
(TANDATANGAN PENULIS) (COP DAN TANDATANGAN PENYELIA)
Alamat Tetap: 28A,
PERSIARAN BERCHAM SELATAN 8, TAMAN DESA KENCANA, 31400 IPOH, PERAK.
(3)
iii
“I hereby declare that this report is the result of my own work except for quotes as cited in the references.”
Signature : ……….
Name : LEE KAM WAH
(4)
iv
“I hereby declare that I have read this report and in my opinion this report is sufficient in terms of the scope and quality for the award of Bachelor of Electronic
Engineering (Computer Engineering) With Honors.”
Signature : ………
Supervisor‟s Name : SYAFEEZA BINTI AHMAD RADZI
: MUZALIFAH BINTI MOHD SAID
(5)
v
Dedication for my family, especially to my beloved father and mother, my lecturers and my friends…
(6)
vi
ACKNOWLEGEMENT
Firstly, I would like to dedicate my highest gratitude to God for giving me the strength to complete this final project.
Secondly, I would like to take this opportunity to express my gratitude to my beloved parents for their continuous support to ensure that I continue growth and success during my educational process. My sincere appreciation also goes to my supervisor, Pn.Syafeeza binti Ahmad Radzi and Pn.Muzalifah binti Mohd Said for giving me prudent advice and guidance in shaping my direction to ensure that I could complete my final project. Thank you for the time and sharing experiences as well as addition knowledge that I believe I would not have this kind of opportunity elsewhere.
Finally I would like to thank also to my entire friends for the support and constructive ideas. Last but not least, I want to thank again to all persons I mentioned above because it was quite hard for me to complete this final project without their guidance, assistance and support.
(7)
vii
ABSTRACT
Overall, the purpose of this project is to build a system that can control electronic appliances by using speech recognition technology. Speech recognition technology is applied in computer systems that convert speech directly into electronic text. This technology allows computer users to bypass their keyboards and use their voices to enter text. It also a fairly natural and intuitive way of controlling the simulation while allowing the user‟s hands to remain free. Basically, this project development consists of three parts. The first part is software development. GUI is used as interface from user to the system. Program code and GUI is designed using Visual Basic 2008 Express while Speech API is used to apply speech recognition in this system. This software provides function to hear voice and access the function of microphone. The second part is using simple circuit as prototype to trigger the output. Simple components are used such as bulb represents a lamp; motor represents a fan and relay represents a door lock. The third part is applying parallel port as a connection between the circuits with the system (personal computer). This speech recognition technology system is operating by input voice from the user and then the system will accept the input. The signal is passing through the parallel port and triggers the output shows at circuit. Otherwise, the output will not trigger and need to give another input.
(8)
viii
ABSTRAK
Tujuan projek ini adalah untuk membina satu system yang boleh mengawal alat-alat elektronik dengan menggunakan teknologi pengakuan pidato. Teknologi pengakuan pidato diterapkan dalam system computer dan menukarkan pidato kepada teks elektronik secara langsung. Teknologi ini membenarkan pengguna computer menggunakan suara untuk memasukkan teks tanpa menggunakan keyboard. Ini juga satu cara yang intuitif dan semulajadi bagi pengguna tanpa menggunakan tangan mereka. Dasarnya, projek ini ada tiga bahagian. Bahagian pertama ialah pembangunan perisian. GUI digunakan sebagai perantaraan antara pengguna dan system. Kod program dan GUI direka dengan Visual Basic 2008 Express manakala Speech API digunakan untuk pengakuan pidato dalam system ini. Perisian ini member fungsi untuk mendengar suara. Bahagian kedua adalah menggunakan litar ringkas sebagai prototaip untuk mencetuskan hasil.Komponen biasa telah digunakan seperti mentol lampu mewakili lampu di rumah, motor mewakili kipas dan relay
mewakili pintu. Bahagian ketiga adalah menggunakan port selari sebagai sambungan antara litar dan system (komputer persendirian). Sistem teknologi pengakuan pidato ini dioperasikan dengan masukan suara dikeluarkan oleh pengguna dan system akan menerimanya. Isyarat akan diarahkan melalui port selari dan mencetuskan hasil yang ditunjukkan dalam litar. Jika tidak, hasil tidak akan dicetuskan dan perlu memberikan masukan yang lain.
(9)
ix
CONTENTS
CHAPTER TITLE PAGE
PROJECT TITLE i
PROJECT STATUS FORM ii
DECLARATION iii
APPROVAL iv
DEDICATION v
ACKNOWLEGEMENT vi
ABSTRACT vii
ABSTRAK viii
CONTENTS ix
LIST OF TABLES xii
LIST OF FIGURES xiii
LIST OF ABBREVIATIONS xv
LIST OF APPENDIXES xvi
I INTRODUCTION
1.1 Introduction 1
1.2 Project Objective 2
1.3 Problem Statement 3
1.4 Scope of Work 3
1.5 Thesis Outline 6
II LITERATURE REVIEW
(10)
x
2.2 Graphical User Interface (GUI) 9
2.3 Parallel Port 10
2.3.1 Hardware Properties 11
2.3.2 Parallel Port Programming in Visual Basic 14
2.3.2.1 How to Output Data 14
2.3.3 Parallel Port versus Serial Port 15
2.4 Windows Speech Recognition 16
2.4.1 Speech API 17
2.4.2 .NET 3.x System.Speech 19
2.5 Microphone 20
2.5.1 Microphone Array 21
2.6 TVicLPT 22
2.6.1 TVicLPT‟s Features 23
III METHODOLOGY
3.1 Introduction 24
3.2 Data Collection Method 25
3.3 Flow Chart of Project Methodology 25
3.4 System Methodology 26
3.5 Speech Recognition Process 27
3.6 Project Phases 28
3.6.1 Graphical User Interface (GUI) Design 29
3.6.2 Programming Code 35
3.7 Circuit 35
IV RESULTS AND ANALYSIS
4.1 Expected Result 37
4.2 Software Result 38
4.3 Software Testing 38
(11)
xi V DISCUSSION AND CONCLUSION
5.1 Discussion 45
5.2 Commercialization 46
5.3 Conclusion 46
5.4 Future Works 46
REFERENCES 48
APPENDIX A 50
APPENDIX B 51
(12)
xii
LIST OF TABLES
NO TITLE PAGE
2.1 Pin Assignments of the D-Type 25 pin Parallel Port Connector 12
(13)
xiii
LIST OF FIGURES
NO TITLE PAGE
1.1 New Windows Application 4
1.2 Speech Recognition in Microsoft Windows 5
1.3 Parallel Port and Connector 6
2.1 Microsoft Visual Basic 2008 Express Edition 9
2.2 Pin Diagram of DB25 Parallel Port 11
2.3 Parallel Port of Computer 12
2.4 A 25-pin Signal Definition for the Parallel Interface 13
2.5 Example Code 14
2.6 Speech APIs in Windows Vista 18
2.7 Module Programming Code in VB .NET 20
2.8 Desktop Microphone 21
2.9 Add TVicLPT Component as Reference 22
3.1 Project Methodology 26
3.2 System Flow Chart 27
3.3 Process of Speech Recognition 28
3.4 Open the Microsoft Visual Basic 2008 Express Edition 29
3.5 Open New Project 30
3.6 Drag the Desired Forms from Toolbox 30
3.7 Change the Form‟s Properties 31
3.8 Form After Changed the Properties 31
3.9 Insert the Picture Box as the Background in Form 32
3.10 Choose an Image 32
3.11 Import an Image 33
(14)
xiv
3.13 Complete the Design of First Form 34
3.14 Design of Second Form 34
3.15 Design of Third Form 35
3.16 Schematic Diagram of Circuit 36
4.1 Expected Result 38
4.2 Login Page 39
4.3 Item Menu 40
4.4 Device On/Off Menu 40
4.5 Microsoft Speech Recognition 41
4.6 Light „On‟ Message Pop Out 41
4.7 Light „Off‟ Message Pop Out 42
4.8 Fan „On‟ Message Pop Out 42
4.9 Fan „Off‟ Message Pop Out 43
4.10 Switch On the Light 43
(15)
xv
LIST OF ABBREVIATIONS
ECP - Extended Capabilities Mode EPP - Extended Parallel Port GUI - Graphical User Interface
IEEE - Institute of Electrical and Electronics Engineers LPT - Line Print Terminal
PC - Personal Computer RSI - Repetitive Strain Injury
SAPI - Speech Application Programming Interface SR - Speech Recognition
TTS - Text-to-Speech VB - Visual Basic
(16)
xvi
LIST OF APPENDIXES
NO TITLE PAGE
A VB Code of Form 1 50
B VB Code of Form 2 51
(17)
CHAPTER I
INTRODUCTION
This chapter explains the background of the project and gives the introduction about the project. This chapter also states the project objectives, problem statements and scope of work.
1.1 Introduction
This project is about developing a Windows application with speech recognition technology to control the electronic and electrical appliances such as fan, lamp, door lock, and the others. This application enables users to control any home appliances by inserting voice commands without the use of hands. This is useful for both able and disabled users. In addition, it is essential for people with limited physical ability.
A computer based system is created to use as an interface to the speech recognition engine which integrated in Microsoft Windows. A software application is developed with graphical user interface (GUI) which designed using Microsoft Visual Basic. Thus, it is easy to be used by everyone. From the software application, it sends an output signal to the hardware circuit through the parallel port. The output signal control the electronic appliances on or off.
(18)
2 Speech recognition is the technology that converts sounds, words or phrases spoken by human to machine-readable input. Speech recognition systems can be trained to recognize specific commands and upon confirmation of correctness instructions can be given to systems. The term “voice recognition” is sometimes used to refer to speech recognition where the recognition system is trained to a particular speaker. Hence there is an aspect of speaker recognition which attempts to identify the person speaking, to better recognize what is being said.
Speech recognition is different from voice recognition, through many people use the terms interchangeably. In the technical sense, voice recognition is strictly about trying to recognize individual voices, not what the speaker said. It is a form of biometrics, the process of identifying a specific individual, often used for security applications. When we type a word or phrase, the computer doesn‟t actually understand English, but it recognizes the command and software tells the computer what to do when that command is recognized.
1.2 Project Objective
The main objective of this project is to design and implement the speech recognition technique to control the electronic and electrical appliances which can ease a person‟s daily activity. This makes the life easier and convenient. Nowadays, many applications can be reached without use of hands such as Bluetooth interact with mobile phone. In addition, it helps to reduce the utility expenses. Other objectives are:
i. To familiar with programming coding using Visual Basic environments. ii. To construct and analyze the suitable circuit for external hardware. iii. To learn and study the specification of parallel port.
iv. To apply the function of parallel port.
(19)
3 1.3 Problem Statement
This project is proposed to solve the problems that occur mostly towards to people with disabilities. The target users are:
i. Handicapped people ii. Paralytic people iii. Elder person
iv. Person who recovered from surgery v. Overweight person
People with disabilities can benefit from speech recognition programs. Speech recognition is especially useful for people who have difficulty using their hands, ranging from mild repetitive stress injuries to involved disabilities that preclude using conventional computer input devices. In fact, people who used the keyboard a lot and developed repetitive strain injury (RSI) became an urgent early market for speech recognition.
By developing this project, it will give them benefit and convenience by lighten their burden. This makes their life easier so that they will not fully depend on other people.
1.4 Scope of Work
This project is started by collecting information about all the requirements which are needed to accomplish these matters. Basically, the searching process begins by exploring information via internet, related books, journals and thesis. Resource from the library and supervisor‟s guidance is used and some essential information is used in this project. After information gathering, the project‟s methodology is being planned. The project development is preceded based on the methodology. Overall, this project is divided into three parts which is consists of:
(20)
4 i. Software writing using Microsoft Visual Basic
Create a windows application which can interface between the speech recognition engine in computer operating system and the hardware circuit such as motor, lamp bulb. The windows application can determine which appliances can switch on or off. There are two main functions, choose the electronic appliances and choose to switch on or off.
Figure 1.1: New Windows Application.
ii. Speech recognition engine from Windows
Users speak commands that are recognized by a piece of software called the speech engine. The speech engine then tells the speech application what the user said, and the application determines what to do next. The speech recognition engine is integrated in Microsoft Windows. We have to be trained by reading text to computer to improve the computer‟s ability to understand our voice. This can help improve dictation accuracy.
(21)
5
Figure 1.2: Speech Recognition in Microsoft Windows.
iii. Hardware
Parallel port is applied to this project. The components used are bulb, motor, relay, resistors, diodes, transistor and the others. The circuit will connect to the parallel port. Computer will send output signals to the circuit through parallel port and control the electronic appliances.
(22)
6
Figure 1.3: Parallel Port and Connector.
1.5 Thesis Outline
This thesis is documentary delivering the ideas generated, concepts applied, activity done, and finally the product itself. It consists of five chapters. The following is the outline for this project in order to understand the whole project.
Chapter I explains briefly about the project background, objective of the project which needs to be achieved, problem statement of the project, scope of works regarding the project and methodology of the project.
Chapter II is a literature review on theoretical concepts applied in this project. It contains the information gathering of the project in order to complete the whole project.
Chapter III introduces the methodology of the project. This chapter introduces the construction of the project, which involves hardware development and software development. The approach for meeting the goals and objectives and project life cycle phase is described in this chapter, along with the tasks needed to complete it.
(23)
7 Chapter IV has covered the entire project finding which includes the simulation design and GUI. This chapter also discusses and analyze about the project and operation of the software such as the programming code. The output from combination of software and hardware is also included.
Chapter V is the conclusion and suggestion to the project for the future recommendation that can be implemented.
(24)
CHAPTER II
LITERATURE REVIEW
This chapter contains the literature review on theoretical concepts applied in this project. It contains the information gathering of the project in order to complete the whole project.
2.1 Microsoft Visual Basic 2008 Express
Visual Basic 2008 is an evolution of the Visual Basic language that is engineered for productively building type-safe and object-oriented applications. Visual Basic enables developers to target Windows, Web, and mobile devices.
Visual Basic Express provides a fully functional development environment for first-time programmers and hobbyists who are interested in building Windows Forms applications, Windows Presentation Foundation client applications, and class libraries. Visual Basic Express is the ideal choice for first-time developers who are interested in learning how to program in the Visual Basic language. If we can imagine a computer program, we can probably create it with Visual Basic Express. From a simple program that displays a message, to a full-fledged application that accesses a database or a Web service, Visual Basic Express gives us the tools we need.
(1)
1.3 Problem Statement
This project is proposed to solve the problems that occur mostly towards to people with disabilities. The target users are:
i. Handicapped people ii. Paralytic people iii. Elder person
iv. Person who recovered from surgery v. Overweight person
People with disabilities can benefit from speech recognition programs. Speech recognition is especially useful for people who have difficulty using their hands, ranging from mild repetitive stress injuries to involved disabilities that preclude using conventional computer input devices. In fact, people who used the keyboard a lot and developed repetitive strain injury (RSI) became an urgent early market for speech recognition.
By developing this project, it will give them benefit and convenience by lighten their burden. This makes their life easier so that they will not fully depend on other people.
1.4 Scope of Work
This project is started by collecting information about all the requirements which are needed to accomplish these matters. Basically, the searching process begins by exploring information via internet, related books, journals and thesis. Resource from the library and supervisor‟s guidance is used and some essential information is used in this project. After information gathering, the project‟s methodology is being planned. The project development is preceded based on the methodology. Overall, this project is divided into three parts which is consists of:
(2)
i. Software writing using Microsoft Visual Basic
Create a windows application which can interface between the speech recognition engine in computer operating system and the hardware circuit such as motor, lamp bulb. The windows application can determine which appliances can switch on or off. There are two main functions, choose the electronic appliances and choose to switch on or off.
Figure 1.1: New Windows Application.
ii. Speech recognition engine from Windows
Users speak commands that are recognized by a piece of software called the speech engine. The speech engine then tells the speech application what the user said, and the application determines what to do next. The speech recognition engine is integrated in Microsoft Windows. We have to be trained by reading text to computer to improve the computer‟s ability to understand our voice. This can help improve dictation accuracy.
(3)
Figure 1.2: Speech Recognition in Microsoft Windows.
iii. Hardware
Parallel port is applied to this project. The components used are bulb, motor, relay, resistors, diodes, transistor and the others. The circuit will connect to the parallel port. Computer will send output signals to the circuit through parallel port and control the electronic appliances.
(4)
Figure 1.3: Parallel Port and Connector.
1.5 Thesis Outline
This thesis is documentary delivering the ideas generated, concepts applied, activity done, and finally the product itself. It consists of five chapters. The following is the outline for this project in order to understand the whole project.
Chapter I explains briefly about the project background, objective of the project which needs to be achieved, problem statement of the project, scope of works regarding the project and methodology of the project.
Chapter II is a literature review on theoretical concepts applied in this project. It contains the information gathering of the project in order to complete the whole project.
Chapter III introduces the methodology of the project. This chapter introduces the construction of the project, which involves hardware development and software development. The approach for meeting the goals and objectives and project life cycle phase is described in this chapter, along with the tasks needed to complete it.
(5)
Chapter IV has covered the entire project finding which includes the simulation design and GUI. This chapter also discusses and analyze about the project and operation of the software such as the programming code. The output from combination of software and hardware is also included.
Chapter V is the conclusion and suggestion to the project for the future recommendation that can be implemented.
(6)
CHAPTER II
LITERATURE REVIEW
This chapter contains the literature review on theoretical concepts applied in this project. It contains the information gathering of the project in order to complete the whole project.
2.1 Microsoft Visual Basic 2008 Express
Visual Basic 2008 is an evolution of the Visual Basic language that is engineered for productively building type-safe and object-oriented applications. Visual Basic enables developers to target Windows, Web, and mobile devices.
Visual Basic Express provides a fully functional development environment for first-time programmers and hobbyists who are interested in building Windows Forms applications, Windows Presentation Foundation client applications, and class libraries. Visual Basic Express is the ideal choice for first-time developers who are interested in learning how to program in the Visual Basic language. If we can imagine a computer program, we can probably create it with Visual Basic Express. From a simple program that displays a message, to a full-fledged application that accesses a database or a Web service, Visual Basic Express gives us the tools we need.