Connectionist Models

3.1 Connectionist Models

In the nineteenth century the associationist approach appeared in psychology. Its representatives claimed that the association of mental states is a basic mechanism of mental processes. In this approach the nature of complex mental phenomena is

© Springer International Publishing Switzerland 2016 23 M. Flasi´nski, Introduction to Artificial Intelligence, DOI 10.1007/978-3-319-40022-8_3

24 3 Computational Intelligence explained by the interaction of simpler ones. This general idea of associationism

was developed by Edward L. Thorndike 1 as the connectionist approach [301]. According to this approach, learning is a result of associations between stimuli and responses to stimuli. Associations become strengthened if an organism is trained with stimulus-response exercises, and they become weakened if such training is discontinued. Responses which are rewarded become strengthened and after some time become habitual responses.

These ideas of connectionism have been assimilated in Artificial Intelligence for the purpose of describing mental processes, which has led to connectionist models in AI. In these models associations are represented with the help of connectionist networks . There are two main types of these networks [21].

In localist connectionist networks each component of knowledge (concept, object, attribute, etc.) is stored in a single element of a network. Therefore, we can include, e.g., semantic networks [21] in this model, although we have ascribed them to sym- bolic AI in a previous chapter. Although they are not treated in AI as typical connec- tionist networks, in fact they fulfill the conditions of their definition. For example, in

the ACT-R model [6], mentioned in Chap. 1 , each node of a semantic network has an activity parameter (a weight), which is used to stimulate strengthening/weakening mechanisms described above. 2 Bayes networks are another example of localist con- nectionist networks. 3 Distributed connectionist networks are the second type of such networks. In this case knowledge is stored in a distributed way, i.e., it is distributed among many elements of a network. Artificial neural networks are the best example of such net- works. According to custom only neural networks are associated with the connec- tionist approach in Artificial Intelligence. Later, our considerations of connectionist models will be limited to distributed connectionist networks only.

In a distributed connectionist approach, mental states are modeled as emergent processes , which take place in networks consisting of elementary processing units. As we have mentioned in Chap. 1 , a process is emergent if it cannot be described on the basis of its elementary sub-processes. This results from the fact that the nature and the functionality of an emergent process is something more than just the simple sum of functionalities of its sub-processes. 4

1 Edward Lee Thorndike—a professor of psychology at Columbia University. His work concerns zoopsychology and educational psychology. He was the President of the American Psychological

Association. 2 In fact, ACT-R is a hybrid AI system, which is based on both the symbolic approach and the

sub-symbolic (CI) approach.

3 Bayes networks are presented in Chap. 12 .

4 Any human mental process is a good example here. Although a single biological neuron does not think, a brain treated as a network consisting of neurons thinks.

3.1 Connectionist Models 25 The fundamentals of distributed connectionism were established by David E.

Rumelhart and James L. McClelland 5 in [253,254]. Apart from the characteristics of this approach mentioned above, we assume that mental states in a network are modeled in such a way that network units process them in a parallel way. The units perform numeric operations. As a consequence of such operations any processing unit can be activated. Then, the result of such an activation is propagated to all units which are connected to this unit. The network acquires and stores knowledge in an implicit way by modifying the parameters (weights) of connections among the processing units. The process of modifying these parameters is treated as network learning .

A model of artificial neural networks as a representative of the (distributed) con-

nectionist approach is presented in detail in Chap. 11 .