Artificial Neural Network Implementation in OCR and Pattern Recognition.

(1)

Abstract

Artificial Neural Network is one of the technologies which have developed because of Information Technology development itself. Nowadays, more and more large companies are implementing Artificial Neural Network to support their needs. Artificial neural networks are a method of information processing and computation that takes benefit of today's technology. Mimicking the processes present in biological neurons, Artificial Neural Networks are used to predict and learn from a given set of data information. At data analysis neural networks are more robust than statistical methods because of their capability to handle small variations of parameters and noise. Through this ability, Artificial Neural Network can predict more accurately.

Keywords: Artificial Neural Network

(2)

Abstrak

Jaringan Saraf Tiruan adalah salah satu teknologi yang berkembang karena pengaruh besar dari perkembangan dunia teknologi informasi itu sendiri. Belakangan ini, semakin banyak perusahaan-perusahaan besar yang mengimplementasikan Artificial Neural Network untuk menunjang keperluan mereka. Artificial Neural Network adalah suatu metoda dari pemrosesan informasi dan komputasi yang mengambil keuntungan dari teknologi terkini. Meniru dari proses pada saraf biologis, Artificial Neural Network dipakai untuk memprediksi dan belajar dari kumpulan data yang telah diberikan sebelumnya. Pada tahap penganalisaan data, Artificial Neural Network lebih tangguh daripada metode statistik tradisional karena kemampuannya untuk mengatasi varian yang kecil dari parameter-parameter dan gangguan. Melalui kemampuan ini, Artificial Neural Network dapat memprediksi dengan lebih akurat.

(3)

PERNYATAAN PUBLIKASI LAPORAN PENELITIAN... 3

PERNYATAAN ORISINALITAS LAPORAN PENELITIAN ... 4

Preface ... 5

Abstract ... 7

Abstrak ... 8

Table of Contents ... 9

Table of Figures ... 10

Table of Tables ... 11

Chapter I: Introduction ... 12

I.1 Problem Occurred ... 12

I.2 Goals ... 12

I.3 Report Boundaries ... 13

Chapter II: Artificial Neural Network Background ... 14

II.1 Artificial Neural Network Definition ... 14

II.2. Historical Background ... 15

II. 3. Advantages and Disadvantages ... 16

II. 4 Choosing Artificial Neural Network... 16

Chapter III: Implementation ... 18

III.1 Analogy to the Brain ... 18

III.2 Artificial Neurons and How They Work ... 19

III.3 Artificial Network Operations ... 21

III.4 Mathematical Model of Artificial Neural Network ... 23

III.5 Training Artificial Neural Network ... 24

III.6 Firing Rule ... 27

III.7 Artificial Neural Network Example / Implementation in Optical Character Recognition ... 28

III.8 Artificial Neural Network Example / Implementation in Pattern Recognition ... 29

III.9 Artificial Neural Network vs. Traditional Computing ... 31

III.9 Applications of Artificial Neural Network... 33

III.10 Artificial Neural Network Vendors ... 35

Chapter IV: Conclusion ... 37

References ... 38

(4)

Table of Figures

Figure 1 Sample Model of Neural Network……….15

Figure 2 An Example of Simple Neuron………..20

Figure 3 Simple Neural Network Diagram……….….21

Figure 4 Mathematical Model of Artificial Neural Network………...…23

Figure 5 Interval Activity Model of A Neuron……….……..…..23

Figure 6 Threshold Function……….……...….24

Figure 7 Piecewise-Linear Function……….……..….24

Figure 8 Sigmoid Function……….……..….24

Figure 9 Supervised Learning Diagram……….…….25

Figure 10 Multilayer Perceptron Method……….………..…….26

Figure 11 Learning Neural Network Diagram……….………..…….26

Figure 12 Demonstration Diagram of OCR using ANN……….………..……….27

Figure 12 Demonstration Diagram of OCR using ANN……….………...29

Figure 13 Input Output Pattern Recognition Example……….……….30

Figure 14 Input Output Example, no. 1……….………..30

Figure 15 Input Output Example, no. 2……….………..31

(5)

Table of Tables

Table 1, Before Firing Rule Applied Table………..27

Table 2, After Firing Rule Applied Table……….28

Table 3, After Firing Rule Applied Table……….30

Table 4, Middle Neuron……….30

Table 5, Bottom Neuron……….30

(6)

Chapter I: Introduction

Artificial Neural Networks are being touted as the sign of the future in computing. They are certainly self learning mechanisms which don’t require the traditional skills of a programmer. But unfortunately, there are a lot of misconceptions among some users who expected that these neuron-inspired processors can do almost anything. These exaggerations have created disappointment among some potential users who have tried, and failed to solve their problems with neural networks. Those applications builders have often come to the conclusion that neural nets are complicated and confusing. Unfortunately that confusion has come from the industry itself. Many of the articles flooding the people appeared touting a large assortment of different neural networks, all with unique claims and specific examples. Currently only a few of these neuron-based paradigms are actually being used commercially. There is one particular structure, the feed forward which is also usually called as bottom-up or top-down, back-propagation network, is by far and away the most popular. Author will explain about this method in chapter 3 of this report. Most of the other neural network structures represent models for thinking that are still being developed in the laboratories. Yet all of these networks are simply tools and as such the only real demand they make is that they require the network architect to learn how to use them.

I.1 Problem Occurred

Being given a massive numbers of data requires a lot of effort to analyze and summarize even for a big company with numerous workers. Companies need a new technology to predict the trend and make decisions according to massive data silos given to be analyzed. Before the Artificial Neural Network era, all of these tasks were given to human or to a conventional programmed application. These methods didn’t give companies accurate predictions like they wanted it to be. With Artificial Neural Network, system can learns and try to give what the result is although there might be imperfections of the result itself.

I.2 Goals

Either humans or other computer techniques use Artificial Neural Network to determine patterns and detect trends that are too complex to be noticed. In the

(7)

category of information has been given to process, a trained neural network can be considered as an “expert”.

The expected goals of Artificial Neural Networks are:

 To predict patterns

 To recognize patterns

 To detect trends

 To make a model of any collected given data

I.3 Report Boundaries

This report was made for academic purpose of fulfilling the task that given to the author as a student at Maranatha Christian University. The author didn’t actually use or experiencing Artificial Neural Network, nor does or tries actual work related to Artificial Neural Network application or program. This report is considered as research paper which is made by doing a research information mainly through internet and websites as references, and author’s knowledge only.

Although this report was made with author’s best effort, but there are still some boundaries available to the fact which the author explained:

 This report explains what Artificial Neural Network is.

 Although Artificial Neural Network was implemented within a company, it is not guaranteed that company would be better than it used to be.

 Artificial Neural Network cannot do everything like user wanted it to be.

 Artificial Neural Network is not designed to solve all the problems within a company directly or indirectly.

 Artificial Neural Network must learn by sample first.

 These writing doesn’t include on how technically Artificial Neural Network works or run in the application or program.

 Artificial Neural Network has the tendency to predict, not given an exact answer.

 This report doesn’t show or explain the mathematical model or formula of Neural Network.

(8)

Chapter IV: Conclusion

In summary, artificial neural networks are one of the promises for the future in computing. They offer an ability to perform tasks outside the scope of traditional processors. They can recognize patterns within vast data sets and then generalize those patterns into recommended courses of action. And another important thing is neural networks learn, they are not programmed.

Yet, even though they are not traditionally programmed, the designing of neural networks does require a skill. It requires an "art." This art involves the understanding of the various network topologies, current hardware, current software tools, the application to be solved, and a strategy to acquire the necessary data to train the network. This art further involves the selection of learning rules, transfer functions, summation functions, and how to connect the neurons within the network.

Then, the art of neural networking requires a lot of hard work as data is fed into the system, performances are monitored, processes tweaked, connections added, rules modified, and on and on until the network achieves the desired results.

These desired results are statistical in nature. The network is not always right. It is for that reason that neural networks are finding themselves in applications where humans are also unable to always be right. Neural networks can now pick stocks, cull marketing prospects, approve loans, deny credit cards, tweak control systems, grade coins, and inspect work.

Yet, the future holds even more promises. Neural networks need faster hardware. They need to become part of hybrid systems which also utilize fuzzy logic and expert systems, even though neural networks have a huge potential we will only get the best of them when they are integrated with computing, AI, fuzzy logic and related subjects.. It is then that these systems will be able to hear speech, read handwriting, and formulate actions. They will be able to become the intelligence behind robots that never wear out nor become distracted. It is then that they will become the leading edge in an age of "intelligent" machines.

(9)

References

Wikimedia Foundation, Inc. http://en.wikipedia.org/wiki/Artificial_neural_network, retrieved June 16, 2009

Learn Artificial Neural Network website http://www.learnartificialneuralnetworks.com/, retrieved June 16, 2009

Steffen Nissen, and all the contributors

http://leenissen.dk/fann/, retrieved June 17, 2009 Muharram J. Panguriseng, etc

http://www.pertamina-ep.com/pdf/hagi_iagi_perhapi_2005.pdf, retrieved June 19, 2009 James Cannady

http://csrc.nist.gov/nissc/1998/proceedings/paperF13.pdf, retrieved June 19, 2009

Yayasan Total Sarana Edukasi http://www.total.or.id/info.php?kk=Neural%20Network, retrieved June 20, 2009

NeuroDimension, Inc. http://www.nd.com/welcome/whatisnn.htm, retrieved June 21, 2009 Prof. Leslie Smith

http://www.cs.stir.ac.uk/~lss/NNIntro/InvSlides.html, retrieved June 22, 2009

The ANNIMAB Society http://www.phil.gu.se/ann/annworld.html, retrieved June 22, 2009 Eyal Reingold

http://www.psych.utoronto.ca/users/reingold/courses/ai/nn.html, retrieved June 22, 2009 Xin Yao