LEARNING IN ARTIFICIAL NEURAL NETWORKS

12.5 LEARNING IN ARTIFICIAL NEURAL NETWORKS

An important consideration in an artificial neural network is the use of an appropriate learning algorithm (or training algorithm). There are hundreds of them. Learning algo-

PART IV INTELLIGENT DECISION SUPPORT SYSTEMS

Learning algorithms

Discrete/binary input

Continuous input

Simple Hopfield

ART-1

Delta rule

ART-3

Outerproduct AM

Carpenter/

Gradient descent

SOFM

Hamming net

Grossberg

Competitive learning Clustering

Nonlinear vs. linear

SOF

ART-

Backpropagation ML

Source: Based on L. Medsker and J. Liebowitz, Design and Development of Expert Systems and Neural Computing, New York: Macmillan, 1994, p.166.

Supervised learning uses a set of inputs for which the appropriate (desired) outputs are known. For example, a historical set of loan applications with the success or failure of the individual to repay the loan has a set of input parameters and presumed known outputs. In one type, the difference between the desired and actual outputs is used to calculate corrections to the weights of the neural network. A variation of this approach simply acknowledges for each input trial whether the output is correct as the network adjusts weights in an attempt to achieve correct results. Examples of this type of learning are backpropagation and the Hopfield network.

In unsupervised learning, only input stimuli are shown to the network. The network is self-organizing; that is, it organizes itself internally so that each hidden processing element responds strategically to a different set of input stimuli (or groups of stimuli). No knowledge is supplied about which classifications (outputs) are correct, and those that the network derives mayor may not be meaningful to the network developer (this is useful for cluster analysis).

However, setting model parameters can control the number of categories into which the network classifies the inputs. Regardless, a human must examine the final categories to assign meaning and determine the usefulness of the results. Examples of this type of learning are adaptive resonance theory (ART) and Kohonen self-organizing feature maps.

157. 671 GENERAL LEARNING PROCESS

CHAPTER 12

ADVANCED INTELLIGENT

SYSTEMS

In supervised learning, the learning process is inductive; that is, connection weights are derived from existing cases. The usual process of learning involves three tasks (Figure 12.11):

1. Compute temporary outputs.

2. Compare outputs with desired targets.

3. Adjust the weights and repeat the process.

When existing outputs are available for comparison, the learning process starts by setting the connection weights, either by rules or randomly. The difference between the actual output (Y or Y t) and the desired output (Z) for a given set of inputs is an error called delta (in calculus, the Greek symbol delta means "difference").

The objective is to minimize the delta (reduce it to 0 if possible), and this is done by adjusting the network's weights. The key is to change the weights in the right direction, making changes that reduce the delta (error). We will show how this is done later.

Information processing with an ANN consists of an attempt to recognize patterns of activities (pattern recognition). During the learning stages, the interconnection weights change in response to training data presented to the system.

Different ANNs compute the delta in different ways, depending on the learning algorithm being used. There are hundreds of learning algorithms for various situations and configurations, some of which are discussed later.