Architecture of the Threshold Parsing Model

6.2.1 Architecture of the Threshold Parsing Model

  A similar model will be applied to the self-paced reading simulation: the Threshold Parsing Model (TPM, see Figure 17, below; and Appendix Seven for the source code). The primary difference between TPM and its progenitor is a variable rate of input that depends on the resources available to the evidence-collecting processors. In brief, the goal of the network is to model the parsing process from the point of ambiguity (the relative pronoun) in a global NP1-NP2-RC ambiguity. The network constitutes a single analysis. The syntactic evidence collector (arriving via nodes S2 in the critical example) favours a Late Closure strategy. It excites node O2 and inhibits O1. If O2 passes the decision threshold first, then Late Closure is the final decision. However, a context- based solution presents itself a moment later. This chapter is most interested with the case when context favours a high attachment. This evidence collector, then, excites O1 and inhibits O2. If the information arrives quickly enough, there is time for the network to turn the tide in favour of O1: the high attachment solution. The network is designed to make higher-span readers favour the context solution and low spans the syntactic solution. Once this decision is reached, the network stops.

  Another difference between the proposed model and its progenitor concerns the weighted connections. The goal of the TPM is well defined and its weights are idealized. The WTA, by contrast, functions as a categorizer where the categories may vary. Thus, the WTA uses the single-layer delta-rule to arrive at its optimal connection

  weights 1 . The TPM does not; its weights are fully excitatory or inhibitory, as shown in Figure 17.

  Key characteristics of the TPM model include:

  • Processing occurs over time; • Syntax is processed faster than context; • Syntax and context information build, reach asymptote, then decay;

  1 In fact, this is not fully implemented in the WTA, but the magnitude term input (with modifications) is precisely equivalent to the output of a feature-detecting single-layer delta-rule network (Wills

  McLaren, 1997).

  • Syntax and context information compete.

  Excitatory connections

  C Inhibitory connections

  O 1

  C 2

  S 1

  O 2

  S 2

  Figure 17: The Threshold Parsing Model (TPM)

  The Input Units The leftmost component of the TPM contains two pairs of input units (see Figure 17).

  The nodes are labelled C or S to indicate a context or syntax evidence collecting mechanism. The number that follows indicates the parsing solution: 1 represents attachment site one (…the servant of the actress who was on the balcony), the high- attaching solution in the ambiguous sentence currently being processed, and 2 represents site two (…the servant of the actress who was on the balcony), the low- attaching solution. Thus, the value of C1 represents the amount of evidence from the context processor for a high attachment, whereas S2 indicates syntactic evidence for low attachment. This evidence changes over time. At time zero, evidence will for any solution be zero; from this point, however, evidence will begin to build. Evidence is represented as an activation vector over nodes C1, C2, S1 and S2. Activation increases The nodes are labelled C or S to indicate a context or syntax evidence collecting mechanism. The number that follows indicates the parsing solution: 1 represents attachment site one (…the servant of the actress who was on the balcony), the high- attaching solution in the ambiguous sentence currently being processed, and 2 represents site two (…the servant of the actress who was on the balcony), the low- attaching solution. Thus, the value of C1 represents the amount of evidence from the context processor for a high attachment, whereas S2 indicates syntactic evidence for low attachment. This evidence changes over time. At time zero, evidence will for any solution be zero; from this point, however, evidence will begin to build. Evidence is represented as an activation vector over nodes C1, C2, S1 and S2. Activation increases

  For any of the computational units in this first stage of processing, activation ranges from 0 to 1. Only nodes that receive input have a positive activation. If a node receives no input, its activation rests at zero. The sigmoid activation function for nodes C1, C2, S1 and S2 is described in (1):

  where i is any given computational node, e is an exponent function, net is the sum input (a single number that ranges from 0 to +1), t is a constant that influences the gradient of the sigmoid, and x is a constant that shifts the entire sigmoid positively along the x-axis. For the TPM, t = 0.1 and x = 5 (the complete parameter list will be given in 6.2.2 Parameters, p. 153). The x constant is an addition that changes the default behaviour of the activation function, where that behaviour was the transformation of a number falling between -1 and +1 to a sigmoid lying from 0 to +1. With the x constant, the input to the function may range between 0 and +1 yet still yield the full sigmoid output.

  The Weighted Connections Nodes C1 to S2 are connected to the output nodes O1 and O2 by eight fixed-weight

  connections. The weights have two values: +1 and -1. Input from a given input node to

  a given output node is the product of the input node’s current activation and multiplied by their shared, weighted connection. For example, if the activation of input node C1 is

  0.6, this is multiplied by 1 to add 0.6 to the net input of O1. But C1 has an equally inhibitory connection to O2. Thus, any given input node will attempt to sway the decision state through both excitation and inhibition.

  The Output Nodes Each output node is recurrent. In addition, each inhibits the other. The total input to a

  unit i on update c is given by: unit i on update c is given by:

  i

  j ≠ i

  where n i,c is the total input to unit i on update c, v c is the total input activation from all

  input nodes multiplied by their weighted connections to output node i, and j is the identity of the competing node. The equation may be expressed verbally: let the total input to the current output unit be the incoming activation from all input units, added to its own activation on the previous update, minus the activation of all other output nodes.

  Given this input, the activation function to output unit i is calculated using either formulae (3) or (4). The first is used when net input is greater than zero, the second when it is less than or equal to zero. The formulae are:

  where n i,c is the total input to unit i on update c and E and D are constants representing the rate of excitation and decay within the unit. In the TPM, E = 0.1 and D = 0.1.