Development Of A Mind Game Using Electroencephalogram (EEG).

(1)

DEVELOPMENT OF A MIND GAME USING ELECTROENCEPHALOGRAM (EEG)

LIM KEAN THENG

This Report Is Submitted In Partial Fulfilment of Requirements for the Bachelor Degree of Electronic Engineering (Computer Engineering)

Faculty of Electronic and Computer Engineering University Technical Malaysia Malacca

(2)

UNIVERSTI TEKNIKAL MALAYSIA MELAKA

FAKULTI KEJURUTERAAN ELEKTRONIK DAN KEJURUTERAAN KOMPUTER

BORANG PENGESAHAN STATUS LAPORAN

PROJEK SARJANA MUDA II

Tajuk Projek : DEVELOPMENT OF A MIND GAME USING ELECTROENCEPHALOGRAM (EEG)

Sesi Pengajian : 1 4 / 1 5

Saya LIM KEAN THENG mengaku membenarkan Laporan Projek Sarjana Muda ini disimpan di Perpustakaan dengan syarat-syarat kegunaan seperti berikut:

1. Laporan adalah hakmilik Universiti Teknikal Malaysia Melaka.

2. Perpustakaan dibenarkan membuat salinan untuk tujuan pengajian sahaja.

3. Perpustakaan dibenarkan membuat salinan laporan ini sebagai bahan pertukaran antara institusi pengajian tinggi.

4. Sila tandakan ( √ ) :

SULIT*

*(Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972)

TERHAD** **(Mengandungi maklumat terhad yang telah ditentukan oleh organisasi/badan di mana penyelidikan dijalankan)

TIDAK TERHAD

Disahkan oleh:

__________________________ ___________________________________

(TANDATANGAN PENULIS) (COP DAN TANDATANGAN PENYELIA)

(3)

iii

“I hereby declare that this report entitled Development of a Mind Game using Electroencephalogram (EEG) is a result of my own work except for the notes that

have been cited clearly in the references.”

Signature :

Name : LIM KEAN THENG Date : 8 JUNE 2015

(4)

“I hereby declared that I have read this report and in my opinion this report is sufficient in terms of the scope and quality for the award of Bachelor of Electronics

Engineering (Computer Engineering) with Honours”

Signature :

Name : DR. LOW YIN FEN Date : 8 JUNE 2015

(5)

To my family for nursing me with love and their dedicated partnership for success in my life.

(6)

ACKNOWLEDGEMENT

This report is the final product to complete my bachelor degree at the University of Technical Malaysia Malacca (UTeM). The project was conducted in the period of September 2014 to June 2015, at the Faculty of Electronic and Computer Engineering. I would like to thank my supervisor, Dr. Low Yin Fen, for valuable discussions and supporting follow-up throughout the project development and for providing me with the necessary equipment. Last but not least, I would like to thank Miss Wan Amirah binti Wazlan for her general support, lectures on EEG equipment and for sharing her thoughts with me.

(7)

vii

ABSTRACT

Electroencephalogram (EEG) based technology are becoming more popular in games design and development as new EEG equipment with portability, stylish wearable and easier usages are available in the market. Such technologies could be implemented in e-learning, entertainment and medical application for the disabled people. This project is to identify the importance of the EEG systems and seek the requirements to make a real-time, manageable and user-friendly BCI for gamers. The main objective of this project was to develop a playable game controlled by EEG signals. This includes building a mind game algorithm that making use of the available EEG device in the market. To achieve the aim, EEG signals were acquired, analyzed and classified. Finally, the classification signal was used to control a real-time feature in the 3D concentration mind game developed by the Orge3D game engine.

This report outlines the step-by-step implementation and testing of this developed game, and the result was a functional 3D game named “spaceship” that can use user EEG signals to control the 3D game features. Three phases have been used to separate EEG signals between imagined movement and relaxation state. In the process, Virtual Reality Peripheral Network (VRPN) server was used as the interface between EEG and the developed game to send the EEG control signals. At the end of this project, a 3D game which can be played according to the user’s EEG signal was developed and designed.

(8)

viii

ABSTRAK

Electroencephalogram (EEG) teknologi menjadi semakin popular dalam reka bentuk permainan dan pembangunan peralatan EEG baru dengan kelebihan mudah alih, kelaziman pakai dan bergaya. Kini, electroencephalogram (EEG) teknologi sebegini boleh didapati di pasaran. Teknologi seperti itu boleh dilaksanakan dalam pembelajaran, hiburan dan aplikasi perubatan untuk orang kurang upaya. Projek ini mengenal pasti kepentingan sistem EEG untuk mereka bentuk sistem permainan yang menarik untuk pemain. Fokus utama projek ini adalah mereka satu permainan yang boleh dimain kawal oleh isyarat EEG. Ini termasuk membina algoritma sistem permainan menggunakan alat EEG yang terdapat di pasaran. Untuk mencapai matlamat ini, isyarat EEG telah diperolehi, dianalisis dan dikelaskan. Akhir sekali, isyarat klasifikasi itu digunakan untuk mengawal entiti yang terdapat dalam permainan yang telah direka guna Orge3D permainan enjin.

Laporan ini menggariskan pelaksanaan dan ujian langkah demi langkah untuk menguji permainan yang direka. Hasilnya, satu sistem permainan 3D berfungsi dinamakan "kapal angkasa" yang boleh menggunakan isyarat EEG pengguna untuk mengawal ciri-ciri permainan 3D telah direka. Tiga fasa telah digunakan untuk memisahkan isyarat EEG antara pergerakan yang dibayangkan dengan keadaaan rehat. Dalam proses ini, satu sistem Realiti Maya Rangkaian (VRPN) telah digunakan sebagai perhubung di antara EEG dengan sistem permainan yang direka untuk menghantar isyarat kawalan EEG. Pada akhir projek ini, satu 3D sistem permainan yang boleh dimain mengikut isyarat EEG pengguna telah dibangunkan dan direka.

(9)

TABLE OF CONTENT

CHAPTER CONTENT PAGE

TITLE i

CONFORMATION REPORT STATUS ii

SUPERVISORS CONFORMATION iii

DECLARATION iv

DEDICATION v

ACKNOWLEDGEMENT vi

ABSTRACT vii

ABSTRAK viii

TABLE OF CONTENT ix

LIST OF TABLE xii

LIST OF FIGURES xiii

LIST OF ATTACHMENTS xv

1 INTRODUCTION 1

1.1 Problem Statement 3

1.2 Objectives 3

1.3 Scope of Work 4

2 LITERATURE REVIEW 5

2.1 Theories of Electroencephalogram (EEG) 5

2.2 Brain Activity Patterns 10

(10)

2.4 Techniques 14

2.4.1 Classification of EEG system 14

2.4.1.1 Emotion 14

2.4.1.2 Movement 15

2.4.2 Signal Processing Algorithms 15

2.4.2.1 Asynchronous vs. Synchronous 15

2.4.2.2 Spatial Filtering 16

2.4.2.3 Frequency and ERP analysis 17 2.4.2.4 Fractal Dimension Model 17

2.4.2.4.1 Higuchi 18

2.4.2.4.2 Box-counting Method 18

2.5 EEG-based Game Design 19

2.5.1 ORGE3D Game Engine 20

2.6 OpenViBE software platform 24

2.6.1 OpenViBE in Touch Approach 25

2.6.2 Brain Activity 25

2.6.3 Virtual Movements 26

3 METHODOLOGY 27

3.1 Project Implementation 27

3.1.1 EEG System Implementation 29

3.1.1.1 Virtual Reality Peripheral Network(VRPN) 30

3.1.2 Game Algorithm Design 30

3.1.3 Signal Processing Technique and Classification 32 3.1.4 System Evaluation and Enhancement 32 3.2 Electroencephalogram (EEG) System Specification 33

4 RESULTS AND DISCUSSION 35

4.1 Game Implementation 36

4.2 Data Acquisition 36

(11)

4.2.1.1 EEG Signal Data for Subject 1 39 4.2.1.2 Beta Band Power Subject 1 40 4.2.1.3 EEG Signal Data for Subject 2 41 4.2.1.4 Beta Band Power Subject 2 42

4.3 Data Processing 43

4.4 Game Control 44

4.5 Discussions 45

5 CONCLUSION AND RECOMMENDATION 50

REFERENCES 53

APPENDIX A 73

APPENDIX B 75

APPENDIX C 77

APPENDIX D 88

APPENDIX E 91

APPENDIX F 104

APPENDIX G 107

APPENDIX H 109

APPENDIX I 130

APPENDIX J 133

APPENDIX K 135

(12)

xii

LIST OF TABLES

NO TITLE PAGE

(13)

xiii

LIST OF FIGURES

NUMBER FIGURE PAGE

2.1 Photo of Hans Berger 5

2.2 Functional Magnetic resonance Imaging (fMRI) 6

2.3 Basic functional brain map 8

2.4 Sample EEG cap technology 9

2.5 Standardized placement of electrodes on scalp 10

2.6 Delta waves 11

2.7 Theta waves 11

2.8 Alpha waves 12

2.9 Beta waves 12

2.10 Gamma waves 12

2.11 ORGE3D frame samples 20

2.12 ORGE3D frame details 21

2.13 ORGE3D main menu 22

2.14 ORGE3D frame examples 23

2.15 Example of EEG signal display platform 24

2.16 Olivier Bertrand photos 25

2.17 OpenViBE system based grid of letters 26

3.1 Flow Chart of the Project Implementation 28

3.2 Block Diagram of EEG System 29

(14)

xiv

3.4 TMSi Porti32 33

3.5 OpenViBE logo 34

3.6 ORGE3D logo 34

4.1 Subject 1 EEG data acquisition 37

4.2 Subject 2 EEG data acquisition 38

4.3 Subject 1 Filtered EEG data of all channels (4-40Hz) 39 4.4 Subject 1 Filtered EEG data of selected channels (4-40Hz) 39 4.5 Subject 1 Raw EEG data of selected channels 39

4.6 Beta Power Spectrum for Subject 1 40

4.7 Subject 2 Filtered EEG data of all channels (4-40Hz) 41 4.8 Subject 2 Filtered EEG data of selected channels (4-40Hz) 41 4.9 Subject 2 Raw EEG data of selected channels 41

4.10 Beta Power Spectrum for Subject 2 42

4.11 Image displayed for relaxation phase 44

4.12 Game Development Diagram 45

4.13 Subject 1 controlling game features 47

(15)

LIST OF ATTACHMENTS

NUMBER TITLE PAGE

A Project Proposal 73

B Project Milestone 75

C OpenViBE Documentation 77

D OpenViBE Designer Algorithm 88

E ORGE3D SDK Coding 91

F 3D developed Game 104

G 3D Game User Manual 107

H EEG Data for 10 Subjects 109

I VRPN Coding 130

J TMSi Porti-32 Specification 133

K INOTEK Poster 135

(16)

CHAPTER 1

INTRODUCTION

Generally, mind games is undertaken for educational aims and sometimes for medical applications besides entertainment. Such game could be a very useful tool for e-learning and discoveries in the medical field. An electroencephalogram (EEG) is the detection test of brain electrical impulses activities using electrodes attached to the scalp. Recently, EEG-based technology is used in neurofeedback games and brain computer interface (BCI). Availability of advanced BCI makes possible for the EEG system to spread over the areas such as digital media, learning, healthcare and other areas of life.

The purpose of this project is to discover the game control possibilities that lie within the EEG field using Brain Computer Interfaces (BCI). The domain of Brain-Computer interfaces (BCI) is a motivating force for implementing EEG-based technology, which is the use of electrodes to record the brain signal activity. The main focus is about developing mind game applications in the entertainment field and helping disabled people to interact with the external world. One of these game applications is the study of electroencephalogram technology (EEG) to help paralyzed

(17)

patients by mapping brain signals to human sensory motor functions. [1] Besides, EEG based game technology development are gaining concentration towards ordinary healthy people, especially gamers. Gamers has become the target group of EEG technology, giving them new experiences and incentive in gameplay. There is a shift of focus not just constrained to treatment for patients, but also entertainment. With this shift, patients could be beneficial as more and more powerful gaming industry get involved, the newly available driver for EEG technology improvements can be designed with better game engines and faster processors.

A phenomenon known as Event-Related Desynchronization (ERD) was utilized in order to use EEG as a control signal. In a relaxed state, the subject’s sensorimotor EEG signals will show a low peak in amplitude range. However, if the subject imagines activity, movement, the sensorimotor EEG signal amplitude will show an increase trend. Subjects can learn to control ERD, which allow the EEG signals to be used as a control signal.

The project was developed based on the target of detecting imagined movement by playing designed game using EEG signals. Basically, there are two different techniques approach for recording brain signals such as invasive and non-invasive technique. An invasive approach requires physical implants of electrodes to measure signals activities such as neuron activities in brains while the non-invasive approach makes use of, for instance, magnetic resonance imaging (MRI) and EEG technology to make measurements. In this project, a non-invasive technique was used in the EEG recording of the electrical potential over the scalp produced by activities of the brain cortex. The game control algorithm design consists of three parts: signal acquisition, signal processing and game implementation. Selective method for training neural network will be used in this project to classify brain wave signals. Other approaches including game algorithm, signal data analysis and results are discussed in this project.

(18)

1.1 Problem statement

There is a lack of EEG based game application for healthy and disabled people. People with severe disabilities, especially to those who suffer from devastating neuromuscular injuries or paralyze which strips their muscular activity cannot perform everyday routine. Individuals who unable to speak and/or use their limbs cannot communicate or manipulating objects. This designed mind game seeks to provide the general medical community discoveries and practical useful clinical application besides entertainment. However, the constraint of the stability of neuronal recordings is difficult to maintain and easy to be interfered by loss making the design difficult to classify the brain waves using neural network. Besides, there is uncertainty about the EEG signal classification in a real-time application. A computational algorithms are needed to reconstruct the activity of large neuronal populations under a range of conditions to allow stable and accurate performance for the BCI interface platform (GUI) to communicate with brain signal.

1.2 Objectives

The main objective of this project is to develop a playable game controlled by brain signals using Electroencephalogram (EEG). To achieve the aim, brain signal activities will be acquired and analyzed. The developed game will be controlled and played by training neural network. The neural network is trained by using brain signal classification with identified techniques.

(19)

1.3 Scope of work

The work will focus on designing of the simple mind game to be played and controlled by using Electroencephalogram (EEG) signals. A processing algorithm will be developed using EEG software tools which can enable high density EEG dataset and provide other brain signal data processing techniques. Different brain signals will be showed by comparing, analyzing and simulating datasets loaded in the software to process the EEG signals. Besides, the software tools will be used to interpret and process the signal activity by making analysis and classification on the EEG signal.

(20)

CHAPTER 2

LITERATURE REVIEW

2.1 Theories of Electroencephalogram (EEG)

Figure 2.1: Photo of Hans Berger (1873-1941) [1]

About 80 years ago, a German scientist named Hans Berger discover electroencephalogram (EEG). Later, new methods had been explored to develop EEG and mainly categorized into two main groups such as Invasive and non-invasive. An invasive method uses electrodes to measure neurons in brain signal which requires physical implants of electrodes in humans or animals [1] [2].

(21)

Figure 2.2: Functional Magnetic resonance Imaging (fMRI) [3]

Every single neuron or local field potentials were made possible to be measured by using this approach. Invasive EEG recordings can be further subdivided into two groups with the types of electrodes used: intraparenchymal recordings and electrocorticogram (ECoG). Intraparenchymal recordings or stereo-EEG obtained using depth electrodes which are stereotactically inserted in neocortical regions. While the ECoG is obtained by implanting electrodes directly on brain surface [3]. Unlike Invasive, a non-invasive approach use EEG based technologies such as Electroencephalography (EEG), Magnetic Resonance Imaging (MRI), functional Magnetic Resonance Imaging (fMRI) and functional near Infrared (fnIR) to record brain activities from scalp with no risk to the subject tested. Figure 2.2 shows the subject undergo experiment using fMRI to record brain activities. Evidence has suggested that a non-invasive way as an alternative methodology for recording of cortical activity, using transmission of cortical potential from the brain surface to scalp electrodes. A non-invasive EEG recordings was used to analyze and detect mental states related to the imagination of limb movements where five healthy subjects has been tested [38] [39] [40].

(22)

Table 2.1: Summary of neuron recording used in Non-invasive approach

Table 2.1 shows the summary of techniques used in the recording neuron activity. There are four techniques shown with physical property used in each approach. The advantages and disadvantages of each technique are discussed and compared as shown in Table 2.1. According to research, the EEG techniques is the most suitable approach as it is less expensive and portable compared to other types of technique such as MEG and fMRI. These techniques require bulky and expensive equipment to record neuron activities. However, there are limitation of the EEG approach as it requires careful placement of electrode directly to the scalp. Compared to fNIR approach, EEG approach has higher temporal resolution enable easier recording of neuron activity [28] [29] [30] [31].

(23)

Figure 2.3: Basic functional brain map [4]

Figure 2.3 shows the basic functional brain map which consists of different lobe such as frontal lobe, parietal lobe, occipital lobe and temporal lobe. The frontal lobes carry out higher mental processes such as thinking, decision making, and planning while parietal lobes can be divided into two functional regions. One involves sensation and perception and the other is concerned with integrating sensory input, primarily with the visual system. The occipital lobe is the visual processing center of the mammalian brain containing most of the anatomical region of the visual cortex. The temporal lobes are involved in processing sensory input into derived meanings for the appropriate retention of visual memories, language comprehension, and emotion association. The primary somatomotor cortex is a brain region that in humans is located in the dorsal portion of the frontal lobe as shown in Figure 2.3. Result shown that motor imagery of upper limbs can be potentially superior using surface EEG recordings by estimating the cortical activity. Recent studies have shown that implanted electrodes of invasive approach allow recording of different brain signals with higher amplitudes and spatial resolution with temporal accuracy than non-invasive EEG approach recording [4].

(24)

Figure 2.4: Sample EEG cap technology [2]

In an EEG, conductive electrodes, made of silver are used to read brain-related electrical potentials from the scalp which place on the head as shown in Figure 2.4. The signal arises from the difference in voltage between the measured electrodes wherever there exists a brain activity. When neurons communicate, current occurs and the simplest event is known as action potential. However, the signal occurred is weak (30-100µV) and has to be amplified. The fast closing and opening of Na+ and K+ ion channels in neuron membrane discharge the potential. The neuron will trigger when membrane depolarize to some thresholds and tracking these discharges over time reveals brain activity [2].

(1)

1.3 Scope of work

(2)

CHAPTER 2

LITERATURE REVIEW

2.1 Theories of Electroencephalogram (EEG)

Figure 2.1: Photo of Hans Berger (1873-1941) [1]

(3)

Figure 2.2: Functional Magnetic resonance Imaging (fMRI) [3]

(4)

Table 2.1: Summary of neuron recording used in Non-invasive approach

(5)

Figure 2.3: Basic functional brain map [4]

(6)

Figure 2.4: Sample EEG cap technology [2]

Development Of A Mind Game Using Electroencephalogram (EEG).

Parts

Dokumen yang terkait

The Parmalim Movement And Its Relations To Si Singa Mangaraja Xii: A Reexamination Of The Development Of Religious Movements In Colonial Indonesia

Mind Controlled Computer Game Using EEG Signals.

Analysis And Development Of A Control Strategy For Robotic Wheelchair Controlled Using Single Channel Eeg Headset.

Development Of Visualisation Suite For A Low Cost Electroencephalography (EEG) System.

Detection Of Entropy Changes In Brain Function Using Electroencephalogram.

GAME DEVELOPMENT âEDUCATION CARDâ BROWSER-BASED USING HTML5.

Analisis Data Rekam EEG pada Beberapa Ko (1)

Development of porous calcium (4)

robot control system Development (4)

A Study of Mobile Robot Control using EEG Emotiv Epoch Sensor

Dukungan

Links

Development Of A Mind Game Using Electroencephalogram (EEG).

Parts

Dokumen yang terkait

The Parmalim Movement And Its Relations To Si Singa Mangaraja Xii: A Reexamination Of The Development Of Religious Movements In Colonial Indonesia

Mind Controlled Computer Game Using EEG Signals.

Analysis And Development Of A Control Strategy For Robotic Wheelchair Controlled Using Single Channel Eeg Headset.

Development Of Visualisation Suite For A Low Cost Electroencephalography (EEG) System.

Detection Of Entropy Changes In Brain Function Using Electroencephalogram.

GAME DEVELOPMENT âEDUCATION CARDâ BROWSER-BASED USING HTML5.

Analisis Data Rekam EEG pada Beberapa Ko (1)

Development of porous calcium (4)

robot control system Development (4)

A Study of Mobile Robot Control using EEG Emotiv Epoch Sensor

Dokumen yang Anda mencari sudah siap untuk unduhkan

GAME DEVELOPMENT âEDUCATION CARDâ BROWSER-BASED USING HTML5.