pothesis Vaadia et al. 1987; Softky and Koch 1993; Ko¨nig et al. 1996; Fujii et al. 1996; Watan-
abe and Aihara 1997; Watanabe et al. 1998. Yet little is known on the dynamics of such neural
networks
composed of
coincidence detector
neurons. Another interesting subject in relation with
temporal spike coding is the active dendrite hy- pothesis Softky 1994. It was originally proposed
to explain the highly irregular firing of cortical cells Softky and Koch 1993 leading to the as-
sumption that neurons work as coincidence detectors.
In this paper, we introduce an associative mem- ory model which utilizes both temporal spike
coding and active dendrite to increase its memory capacity. In the proposed model, we use a neuron
model which integrates its postsynaptic potentials in a multi-dimensional phase space composed of
time and dendritic space. Here, dendrites are as- sumed to be active and action potentials are gen-
erated according to the local summation of postsynaptic potentials. Moreover, we decide the
spike propagation delay and the synaptic site on the dendrite so that the postsynaptic potentials
are localized in the time-dendritic phase space for each pattern. As a result, the signal to noise ratio
is improved during retrieval leading to increased memory capacity.
2. Model
2
.
1
. Network model We introduce a neuron model which integrates
its postsynaptic potential in a multi-dimensional phase space of time and dendritic space. For the
sake of simplicity, we assume only one dimension for dendritic space, resulting in a two-dimensional
phase space. A postsynaptic potential is given as a curved surface on the two-dimensional phase
space Fig. 1. We also assume that postsynaptic potential increases stepwise at the time of spike
arrival and decays exponentially both in time and space.
Before giving equations and going into the de- tails of the model, we take up two assumptions
which we make to construct a simple model. The first is that neuronal firing is localized in time. We
realize this by setting the variance of spike propa- gation delay small enough so that spikes sent
from a temporally localized group of firing neu- rons do not spread out in time and do not overlap
with spikes sent out by the next temporally local- ized neuronal firing resulting from the current.
We call this group of spikes, a ‘spike group’. The second assumption is to fix the number of firing
neurons resulting from a single spike group. This idea was introduced by Amari 1989 as ‘active
threshold’assuming some kind of a fast negative feedback loop which keeps a moderate rate of
activity in the network.
The summation of the postsynaptic potentials for neuron i in the k + 1th spike group at time t
and dendritic space r is given as the following equation:
I
i
k + 1, r, t =
N j = 1
w
ij
x
j
k, t − d
ij
exp − t − t
k, j f
− d
ij
t exp −
r−s
ij
D, 1
where, w
ij
, d
ij
and s
ij
denote the synaptic weight, spike propagation delay and the synaptic site
from neuron j to neuron i, respectively. Moreover, t
and D denote the time and spatial decay con- stant of the postsynaptic potential and N the total
number of neurons in the network. Finally,
Fig. 1. Postsynaptic potential in a two-dimensional phase space of time and dendritic space. PSP rises stepwise at the
time of spike arrival and decays exponentially both in time and dendritic space.
x
j
k, t is the output of the jth neuron in the kth spike group given as,
If not choosed to fire in the kth spike group x
j
k, t = 0, 2
If choosed to fire in the kth spike group x
j
k, t = t B t
k, j f
1 t ] t
k, j f
3 where t
k, j f
denotes the time of neuronal firing in the kth spike group of the jth neuron.
From the integrated landscape of postsynaptic potentials in the phase space Eq. 1 we choose
the peak point. We denote the coordinates of this maximum point as t
k + 1,i max
, r
k + 1,i max
, with the value I
k + 1,i max
. By assuming an active threshold Amari 1989, we choose the neurons which have the aN
highest peak values I
k + 1,i max
to fire at time t
k + 1,i max
. Here, a is the network activity which determines
the ratio of neurons to fire. One important feature of our network model is that, by putting t
and D , it becomes equivalent to the original sparse coding model by Amari 1989.
2
.
2
. Rules for synaptic site and spike propagation delay
In this subsection, we give the methods to determine w
ij
, d
ij
and s
ij
. First, the synaptic weight w
ij
is given as the conventional autocorrelation associative memory Nakano 1972; Anderson
1972; Kohonen 1972; Hopfield 1982, w
ij
=
m m =
1
p
i m
p
j m
, w
ii
= 0,
4 to embed m random patterns p
m
, where p
i m
is the ith component of p
m
which takes binary values either 0 or 1. Each p
m
have exactly aN compo- nents which take 1 and 1 − aN components
which take 0. Next, spike propagation delay d
ij
and synaptic site s
ij
is given as, d
ij
= d
max
− d
min
m × m
min
i, j + d
min
5 s
ij
= s
max
− s
min
m × m
max
i, j + s
min
6
Fig. 2. The effect of pattern inherent spike propagation delay and synaptic site. U
1
denotes the group of firing neurons that are included in the retrieved pattern, whereas U
2
denotes the group of firing neurons that are not. Left hand figures gives
the neuronal firing in the previous spike group. Right upper and right lower phase space gives the distribution of spike
arrival for neurons belonging to U
1
and U
2
, respectively.
where d
max
, d
min
and s
max
, s
min
give the ranges of the delay and the synaptic site, respectively.
Moreover, m
max
i, j and m
min
i, j denote the maximum and minimum of pattern index m where
p
i m
p
j m
= 1 is satisfied. By determining d
ij
and s
ij
as in the above equations, patterns will likely to have
its inherent delay and synaptic site. Consequently, spikes to neurons which should fire, namely, the
neurons which compose the previous retrieving pattern, will likely to gather in a localized area in
the two-dimensional phase space of the time-den- dritic space Fig. 2. On the other hand, for the
neurons which should not fire, incident spikes will have no correlations and will be dispersed in the
phase space. As a result, the signal to noise ratio is improved, leading to increased memory capac-
ity as seen in the next section.
3. Simulation results
