How to Describe Neuronal Activity: 
Spikes, Rates, or Assemblies? 
Wulfram Getstrier and J. Leo van Hemmen 
Physik-Department der TU Miinchen 
D-85748 Garching bei Miinchen, Germany 
Abstract 
What is the 'correct' theoretical description of neuronal activity? 
The analysis of the dynamics of a globally connected network of 
spiking neurons (the Spike Response Model) shows that a descrip- 
tion by mean firing rates is possible only if active neurons fire in- 
coherently. If firing occurs coherently or with spatio-temporal cor- 
relations, the spike structure of the neural code becomes relevant. 
Alternatively, neurons can be gathered into local or distributed en- 
sembles or 'assemblies'. A description based on the mean ensemble 
activity is, in principle, possible but the interaction between differ- 
ent assemblies becomes highly nonlinear. A description with spikes 
should therefore be preferred. 
1 INTRODUCTION 
Neurons communicate by sequences of short pulses, the so-called action potentials 
or spikes. One of the most important problems in theoretical neuroscience concerns 
the question of how information on the environment is encoded in such spike trains: 
Is the exact timing of spikes with relation to earlier spikes relevant (spike or interval 
code (MacKay and McCulloch 1952) or does the mean firing rate averaged over sev- 
eral spikes contain all important information (rate code; see, e.g., Stein 1967)7 Are 
spikes of single neurons important or do we have to consider ensembles of equivalent 
neurons (ensemble code)7 If so, can we find local ensembles (e.g., columns; Hubel 
and Wiesel 1962) or do neurons form 'assemblies' (Hebb 1949) distributed all over 
the network7 
463 
464 Gerstner and van Hemmen 
2 SPIKE RESPONSE MODEL 
We consider a globally connected network of N neurons with 1 _< i < N. A neuron i 
fires, if its membrane potential passes a threshold 0. A spike at time t[ is described 
by a &pulse; thus Si(t) F 
---- Ef=i ((t -- t/f) is the spike train of neuron i. Spikes are 
labelled such that t is the most recent spike and t/F is the Ftn spike going back in 
time. 
In the Spike Response Model, short SRM, (Gerstner 1990, Gerstner and van Hem- 
men 1992) a neuron is characterized by two different response functions, e and l ref. 
Spikes which neuron i receives from other neurons evoke a synaptic potential 
N o 
j--1 
(1) 
where the response kernel 
: s-a' exp( 
T s Ts 
for s < Atr 
for s > Atr (2) 
describes a typical excitatory or inhibitory postsynaptic potential; see Fig. 1. The 
weight Zij is the synaptic efficacy of a connection from j to i, Atr is the axonal (and 
synaptic) transmission time, and rs is a time constant of the postsynaptic neuron. 
The origin s = 0 in (2) denotes the firing time of a presynaptic spike. In simulations 
we usually assume rs = 2 ms and for Atr a value between 1 and 4 ms 
Similarly, spike emission induces refractoriness immediately after spiking. This is 
modelled by a refractory potential 
h? (t) = (s)S 1 (t - s)d 
(3) 
with a refractory function 
{-cx for S < 7 re 
rf(s) = r/o/(S-7 ) for s  7 r'. 
(4) 
For 0 _ s _< 7 ! the neuron is in the absolute refractory period and cannot spike at 
all whereas for s > 7 ! spiking is possible but difficult (relative refractory period). 
To put it differently, 0 - r/ (s) describes an increased threshold immediately after 
spiking; cf. Fig. 1. In simulations, 7 ! is taken to be 4 ms. Note that, for the sake 
of simplicity, we assume that only the most recent spike $/ induces refractoriness 
whereas all past spikes $ contribute to the synaptic potential; cf., Eqs. (1) and (3). 
How to Describe Neuronal Activity: Spikes, Rates, or Assemblies? 465 
1,0 
0.0 
0.0 5.0 
10.0 15.0 20.0 
s[ms] 
Fig I Response functions. 
Immediately after firing at s = 
0 the effective threshold is in- 
creased to 0-r/f(s)(dashed). 
The form of an excitatory post- 
synaptic potential (EPSP) is 
described by the response func- 
tion e(s) (sohd). It is delayed by 
a time A tr. The arrow denotes 
the period Toc of coherent os- 
cillations; cf. Section 5. 
The total membrane potential is the sum of both parts, i.e. 
h,(t) = ? (t) + 
(5) 
Noise is included by introduction of a firing probability 
PF(h; St) = r-l (h ) St. (6) 
where 5t is an infinitesimal time interval and r(h) is a time constant which depends 
on the momentary value of the membrane potential in relation to the threshold 0. 
In analogy to the chemical reaction constant we assume 
r(h) = r0 exp[-/3(h - 0)], (7) 
where r0 is the response time at threshold. The parameter/3 determines the amount 
of noise in the system. For/3 - oo we recover the noise-free behavior, i.e., a neuron 
fires immediately, if h > 0 (r - 0), but it cannot fire, if h < 0 (r - oo). Eqs. (1), 
(3), (5), and (6) define the spiking dynamics in a network of SRM-neurons. 
3 FIRING STATISTICS 
We start our considerations with a large ensemble of identical neurons driven by the 
same arbitrary synaptic potential hSU"(t). We assume that all neurons have fired a 
first spike at t = tl . Thus the total membrane potential is h(t) = hY"(t) + rl rf (t- 
tl). If h(t) slowly approaches 0, some of the neurons will fire again. We now ask 
for the probability that a neuron which has fired at time t{ will fire again at a later 
time t. The conditional probability P?)(tlt{) that the next spike of a given neuron 
occurs at time t > t{ is 
P(F2)(tltl ) = r-l[h(t)]exp - r-[h(s')]ds ' . (8) 
The exponential factor is the portion of neurons that have survived from time t{ to 
time t without firing again and the prefactor r-l[h(t)] is the instantaneous firing 
probability (6) at time t. Since the refractory potential is reset after each spike, 
the spiking statistics does not depend on earlier spikes, in other words, it is fully 
described by P?)(tlt). This will be used below; cf. Eq. (14). 
466 Gerstner and van Hemmen 
As a special case, we may consider constant synaptic input h y" -- h . In this case, 
(8) yields the distribution of inter-spike intervals in a spike train of a neuron driven 
by constant input h . The mean firing rate at an input level h  is defined as the 
inverse of the mean inter-spike interval. Integration by parts yields 
f[h ] -- dt(t-t)P?)(tltl ) = ds exp { 7'-l[hOLvref(st)ld8 t } 
(9) 
Thus both firing rate and interval distribution can be calculated for arbitrary inputs. 
4 
ASSEMBLY FORMATION AND NETWORK 
DYNAMICS 
We now turn to a large, but structured network. Structure is induced by the 
formation of different assemblies in the system. Each neuronal assembly cu (Hebb 
1949) consists of neurons which have the tendency to be active at the same time. 
Following the traditional interpretation, active means an elevated mean firing rate 
during some reasonable period of time. Later, in Section 5.3, we will deal with a 
different interpretation where active means a spike within a time window of a few 
ms. In any case, the notion of simultaneous activity allows to define an activity 
pattern {, 1 _< i <_ N} with  = i if i G cu and  = 0 otherwise. Each neuron 
may belong to different assemblies 1 _< tt <_ q. The vector i = (/,...,[) is the 
'identity card' of neuron i, e.g., i = (1, 0, 0, 1, 0) says that neuron i belongs to 
assembly i and 4 but not to assembly 2,3, and 5. 
Note that, in general, there are many neurons with the same identity card. This 
can be used to define ensembles (or sublattices) L(x) of equivalent neurons, i.e., 
L(x) = {ili = x} (van Hemmen and Kiihn 1991). In general, the number of 
neurons IL(x)l in an ensemble L(x) goes to infinity if N - cx:, and we write 
IL(x)l = p(x)N. The mean activity of an ensemble L(x) can be defined by 
et+At 
A(x,t)= lim lim IL(x)l- x J, Sii(t)dt. (10) 
At--*0 
i ) 
In the following we assume that the synaptic efficacies have been adjusted according 
to some Hebbian learning rule in a way that allows to stabilize the different activity 
patterns or assemblies a u. To be specific, we assume 
J0 q q 
Jij -- - Y Y Qupost()pre() (11) 
/=1 ':1 
where post(x) and pre(x) are some arbitrary functions characterizing the pre- and 
postsynaptic part of synaptic learning. Note that for Qu = u and post(x) and 
pre(x) linear, Eq. (11) can be reduced to the usual Hebb rule. 
With the above definitions we can write the synaptic potential of a neuron i  L(x) 
in the following form 
q q oo 
h*Y"x,t) = Jo Y. Y] Qupost(x ) y.pre(z ) fo (s')p(z)A(z,t - s')ds'. (12) 
/=1 ,=1 
How to Describe Neuronal Activity: Spikes, Rates, or Assemblies? 467 
We note that the index i and j has disappeared and there remains a dependence 
upon x and z only. The activity of a typical ensemble is given by (Gerstner and 
van Hemmen 1993, 1994) 
A(x, t): P?)(tlt- s)A(x,t- s)ds (13) 
where 
p?)(tlt_s)=r-l[hY"(x,t)+rf*/(s)]exp- ' 
(14) 
is the conditional probability (8) that a neuron i  L(x) which has fired at time 
t-s fires again at time t. Equations (12) - (14) define the ensemble dynamics of the 
network. 
5 DISCUSSION 
5.1 ENSEMBLE CODE 
Equations. (12) - (14) show that in a large network a description by mean ensemble 
activities is, in principle, possible. A couple of things, however, should be noted. 
First, the interaction between the activity of different ensembles is highly nonlinear. 
It involves three integrations over the past and one exponentiation; cf. (12) - (14). 
If we had started theoretical modeling with an approach based on mean activities, 
it would have been hard to find the correct interaction term. 
Second, L(x) defines an ensemble of equivalent neurons which is a subset of a given 
assembly cu. A reduction of (12) to pure assembly activities is, in general, not 
possible. Finally, equivalent neurons that form an ensemble L(x) are not necessarily 
situated next to each other. In fact, they may be distributed all over the network; 
cf. Fig. 2. In this case a local ensemble average yields meaningless results. A 
theoretical model based on local ensemble averaging is useful only if we know that 
neighboring neurons have the same 'identity card'. 
a) activity 
 20 
1 oo 
b) o) rate [Hz] 
=1 2o 
 10 
0 
100 
1 $0 200 
time [ms] 
'.'.'-'.';:: : .':'-: ,:;'.:: C C . 
:.:.'.. :.', 
'.'  '.'  :' :' ..:.? .. =..' ..-..-....: .. 
1 $0 200 
time [ms] 
0 1 O0 200 
rate [Hz] 
Fig. 2 
Stationary activity (incoherent 
firing). In this case a descrip- 
tion by firing rates is possible. 
(a) Ensemble averaged activity 
A(x,t). (b) Spike raster of 30 
neurons out of a network of 
4000. (c) Time-averaged mean 
firing rate f. We have two dif- 
ferent assemblies, one of them 
active (A t = 2 ms, /3 = 5). 
5.2 RATE CODE 
Can the system of Eqs. (12) -(14) be transformed into a rate description? In general, 
this is not the case but if we assume that the ensemble activities are constant in 
468 Gerstner and van Hemmen 
1.O 
0.8 
0.6 
0.4 
0.2 
0.0 
0 I O0 
200 300 
Zeit [ms] 
A x 35 
A 4 
400 500 600 
Fig. 3 Stability of stationary states.The postsynaptic potential h u" is plotted as a function 
of time. Every 100 ms the delay A t has been increased by 0.5 ms. In the stationary state 
(A t = 1.5 ms and A t = 3.5 ms), active neurons fire regularly with rate T - = 1/5.5 ms. 
For a delay A t > 3.5 ms, oscillations with period Wl = 2r/Tp build up rapidly. For 
intermediate delays 2 _ A t _ 2.5 small-amplitude oscillations with twice the frequency 
occur. Higher harmonics are suppressed by noise (/3 = 20). 
time, i.e., A(x,t) -- A(x), then an exact reduction is possible. The result 
fixed-point equation (Gerstner and van Hemmen 1992) 
q q 
A(x) = f[Jo E E Oupst(xu) Epre(z)P(z)A(z)] 
/=1 '-1 Z 
where 
{/o /o 
f[h 'u'] = dsexp{- r-[h u' + rlref(st)]dst} 
is a 
(15) 
is the mean firing rate (9) of a typical neuron stimulated by a synaptic input h u. 
Constant activities correspond to incoherent, stationary firing and in this case a 
rate code is sufficient; cf. Fig. 2. 
Two points should, however, be kept in mind. First, a stationary state of incoherent 
firing is not necessarily stable. In fact, in a noise-free system the stationary state 
is always unstable and oscillations build up (Gerstner and van Hemmen 1993). In 
a system with noise, the stability depends on the noise level fi and the delay Atr of 
axonal and synaptic transmission (Gerstner and van Hemmen 1994). This is shown 
in Fig. 3 where the delay A t has been increased every 100 ms. The frequency of 
the smMl-amplitude oscillation around the stationary state is approximately equal 
to the mean firing rate (16) in the stationary state or higher harmonics thereof. 
A smMl-amplitude oscillation corresponds to partially synchronized activity. Note 
that for A t = 4 ms a large-amplitude oscillation builds up. Here all neurons fire in 
nearly perfect synchrony; cf. Fig. 4. In the noiseless case/? - oc, the oscillations 
period of such a collective or 'locked' oscillation can be found from the threshold 
condition 
Toc=inf s[O=rf*f(s)+Jo c(ns) . (17) 
n----1 
In most cases the contribution with n = 1 is dominant which allows a simple graph- 
ical solution. The first intersection of the effective threshold 0- r/J'(s) with the 
(16) 
How to Describe Neuronal Activity: Spikes, Rates, or Assemblies? 469 
weighted EPSP Joe(s) yields the oscillation period; cf. Fig 1. An analytical argu- 
d e 
ment shows that locking is stable only if  12o,c > 0 (Gerstner and van Hemmen 
1993). 
a) activity 
b) 
o 
1 oo 1,50 200 
time [ms] 
30 
 10 
0 
100 
rate [Hz] 
0 100 200 
rate [Hz] 
Fig. 4 
Oscillatory activity (coherent 
firing). In this case a descrip- 
tion by firing rates must be com- 
bined with a description by en- 
semble activities. (a) Ensemble 
averaged activity A(x,t). (b) 
Spike raster of 30 neurons out 
of a network of 4000. (c) Time- 
averaged mean firing rate f. In 
this simulation, we have used 
A r=4ms and/?=8. 
Second, even if the incoherent state is stable and attractive, there is always a transi- 
tion time before the stationary state is assumed. During this time, a rate description 
is insufficient and we have to go back to the full dynamic equations (12) - (14). Sim- 
ilarly, if neurons are subject to a fast time-dependent external stimulus, a rate code 
fails. 
5.3 SPIKE CODE 
A superficial inspection of Eqs. (12) - (14) gives the impression that all information 
about neuronal spiking has disappeared. This is, however, false. The term A(x, t-s) 
in (13) denotes all neurons with 'identity card' x that have fired at time t-s. The 
integration kernel in (13) is the conditional probability that one of these neurons 
fires again at time t. Keeping t-s fixed and varying t we get the distribution 
of inter-spike intervals for neurons in L(x). Thus information on both spikes and 
intervals is contained in (13) and (14). 
We can make use of this fact, if we consider network states where in every time step a 
different assembly is active. This leads to a spatio-temporal spike pattern as shown 
in Fig. 5. To transform a specific spike pattern into a stable state of the network 
we can use a Hebbian learning rule. However, in contrast to the standard rule, a 
synapse is strenthened only if pre- and postsynaptic activity occurs simultaneously 
within a time window of a few ms (Gerstner et al. 1993). Note that in this case, 
averaging over time or space spoils the information contained in the spike pattern. 
5.4 CONCLUSIONS 
Equations. (12) - (14) show that in our large and fully connected network an 
ensemble code with an appropriately chosen ensemble is sufficient. If, however, the 
efficacies (11) and the connection scheme become more involved, the construction 
of appropriate ensembles becomes more and more difficult. Also, in a finite network 
we cannot make use of the law of large number in defining the activities (10). Thus, 
in general, we should always start with a network model of spiking neurons. 
470 Gerstner and van Hemmen 
a) activity 
b) 
2O 
100 150 200 
time [ms] 
30 '  'e ' ' "' ' ' ' 
..... 
(oo 15o 200 
time [ms] 
rate [Hz] 
100 200 
rate [Hz] 
Fig. 5 
Spatio-temporal spike pattern. 
In this case, neither firing rates 
nor locally averaged activities 
contain enough information to 
describe the state of the net- 
work. (a) Ensemble averaged 
activity A(t). (b) Spike raster of 
30 neurons out of a network of 
4000. (c) Time-averaged mean 
firing rate f. 
Acknowledgements: This work has been supported by the Deutsche Forschungs- 
gemeinschaft (DFG) under grant No. He 1729/2-1. 
References 
Gerstner W (1990) Associative memory in a network of 'biological' neurons. In: 
Advances in Neural Information Processing Systems 3, edited by R.P. Lippmann, 
J.E. Moody, and D.S. Touretzky (Morgan Kaufmann, San Mateo, CA) pp 84-90 
Gerstner W and van Hemmen JL (1992a) Associative memory in a network of 
'spiking' neurons. Network 3:139-164 
Gerstner W, van Hemmen JL (1993) Coherence and incoherence in a globally cou- 
pled ensemble of pulse-emitting units. Phys. Rev. Lett. 71:312-315 
Gerstner W, Ritz R, van Hemmen JL (1993b) Why spikes? Hebbian learning and 
retrieval of time-resolved excitation patterns. Biol. Cybern. 69:503-515 
Gerstner W and van Hemmen JL (1994) Coding and Information processing in 
neural systems. In: Models of neural networks, Vol. 2, edited by E. Domany, J.L. 
van Hemmen and K. Schulten (Springer-Verlag, Berlin, Heidelberg, New York) pp 
lff 
Hebb DO (1949) The Organization of Behavior. Wiley, NewYork 
van Hemmen JL and Kiihn R(1991) Collective phenomena in neural networks. In: 
Models of neural networks, edited by E. Domany, J.L. van Hemmen and K. Schulten 
(Springer-Verlag, Berlin, Heidelberg, New York) pp lff 
Hubel DH, Wiesel TN (1962) Receptive fields, binocular interaction and functional 
architecture in the cat's visual cortex. J. Neurophysiol. 28:215-243 
MacKay DM, McCulloch WS (1952) The limiting information capacity of a neuronal 
link. Bull. of Mathm. Biophysics 14:127-135 
Stein RB (1967) The information capacity of nerve cells using a frequency code. 
Biophys. J. 7:797-826 
