Burst Synchronization Without 
Frequency-Locking in a Completely Solvable 
Network Model 
Heinz Schuster 
Institut f/Jr theoretische Physik 
Universitt Kiel 
Olshausenstraie 40 
2300 Kiel 1, Germany 
Christof Koch 
Computation and Neural System Program 
California Institute of Technology 
Pasadena, California 91125, USA 
Abstract 
The dynamic behavior of a network model consisting of all-to-all excitatory 
coupled binary neurons with global inhibition is studied analytically and 
numerically. We prove that for random input signals, the output of the 
network consists of synchronized bursts with apparently random intermis- 
sions of noisy activity. Our results suggest that synchronous bursts can be 
generated by a simple neuronal architecture which amplifies incoming coin- 
cident signals. This synchronization process is accompanied by dampened 
oscillations which, by themselves, however, do not play any constructive 
role in this and can therefore be considered to be an epiphenomenon. 
I INTRODUCTION 
Recently synchronization phenomena in neural networks have attracted considerable 
attention. Gray e al. (1989, 1990) as well as Eckhorn e al. (1988) provided 
electrophysiological evidence that neurons in the visual cortex of cats discharge in a 
semi-synchronous, oscillatory manner in the 40 Hz range and that the firing activity 
of neurons up to 10 mm away is phase-locked with a mean phase-shift of less than 
3 msec. It has been proposed that this phase synchronization can solve the binding 
problem for figure-ground segregation (yon der Malsburg and Schneider, 1986) and 
underly visual attention and awareness (Crick and Koch, 1990). 
A number of theoretical explanations based on coupled (relaxation) oscillator mod- 
117 
1 18 Schuster and Koch 
els have been proposed for burst synchronization (Sompolinsky et ak, 1990). The 
crucial issue of phase synchronization has also recently been addressed by Bush and 
Douglas (1991), who simulated the dynamics of a network consisting ofbursty, layer 
V pyramidal cells coupled to a common pool of basket cells inhibiting all pyramidal 
cells.  Bush and Douglas found that excitatory interactions between the pyramidal 
cells increases the total neural activity as expected and that global inhibition leads 
to synchronized bursts with random intermissions. These population bursts appear 
to occur in a random manner in their model. The basic mechanism for the observed 
burst synchronization is hidden in the numerous anatomical and biophysical details 
of their model. These, and the related observation that to date no strong oscilla- 
tions have been recorded in the neuronal activity in visual cortex of awake monkeys, 
prompted us to investigate how phase synchronization can occur in the absence of 
frequency locking. 
2 A COINCIDENCE NETWORK 
We consider n excitatory coupled binary McCulloch-Pitts (1943) neurons whose 
output x i  [0, 1] at time t + 1 is given by: 
i = o' zi + i - 0 (1) 
i 
Here win > 1 is the normalized excitatory all-to-all synaptic coupling,  represents 
the external binary input and a[z] is the Heaviside step function, such that a[z] = 1 
for z > 0 and 0 elsewhere. Each neuron has the same dynamic threshold 0 > 0. 
Next we introduce the fraction m t of neurons which fire simultaneously at time t: 
_ t 
mt = I  xi (2) 
i 
In general, 0 < m t < 1; only if every neuron is active at time t do we have ra t -- 1. 
By summing eq. (1) we obtain the following equation of motion for our simple 
network. 
mr+, = 1 Z '[wmt +  -- 01 (3) 
i 
The behavior of this (n+l)-state automata is fully described by the phase-state 
diagram of Figure 1. If 0 > 1 and 0/w > 1, the output of the network m t will 
vary with the input until at some time t', mt' = 0. Since the threshold 0 is always 
larger than the input, the network will remain in this state for all subsequent times. 
If 0 < 1 and O/w < 1, the network will drift until it comes to the state mt' = 1. 
Since subsequent wm t is at all times larger than the threshold, the network remains 
latched at m t = 1. If 0 > 1, but 0/w < 1, the network can latch in either the 
m t = 0 or the m t = 1 state and will remain there indefinitely. Lastly, if 0 < 1, but 
0/w > 1, the threshold is by itself not large enough to keep the network latched 
 This model bears similarities to Wilson and Bower's (1992) model describing the origin 
of phase-locking in olfactory cortex. 
Burst Synchronization without Frequency Locking in a Completely Solvable Network Model 119 
Weight w 
1 
0 
Threshold 0 
Figure 1: Phase diagram for the network described by eq. (3). Different regions 
correspond to different stationary output states m t in the long time limit. 
into the m t = 1 state. Defining the normalized input activity as 
st 1  t 
- n i (4) 
i 
with 0 _< s t _< 1, we see that in this part of phase space m t+ 
activity faithfully reflects the input activity at the previous time step. 
and the output 
with 
m,+Z = 1  r[wm'-t-- O(m')] 
i 
(5) 
O(mt )= { 0< 1 form t < 1 
>co+l for mt= 1 
Therefore, we are operating in the topmost left part of Fig. 1 but preventing the 
network from latching to m t = 1 by resetting it. Such a dynamic threshold bears 
some similarities to the models of Horn and Usher (1990) and others, but is much 
simpler. Note that 0(m t) exactly mimics the effect of a common inhibitory neuron 
which is only excited if all neurons fire simultaneously. 
Our network now acts as a coincidence detector, such that all neurons will "fire" 
at time t + 2, i.e., rn] +2 = 1 if at least k neurons receive at time t a "1" as input. 
k is the smallest integer with k > 0  n/co. The threshold 0(m t) is then transiently 
increased and the network is reset and the game begins anew. In other words, the 
Let us introduce an adaptive time-dependent threshold, 0 t. We assume that 0 t 
remains at its value 0 < 1 as long as the total activity remains less than 1. If, 
however, m t = 1, we increase 0 t to a value larger than co + 1. This has the effect of 
resetting the activity of the entire network to 0 in the next time step, i.e., m + = 
(l/n) Y-i r(co + i - (co + 1 + e)) = 0. The threshold will then automatically reset 
itself to its old value: 
120 
Schuster and Koch 
Output 
0.8, 
0.6 
O.,l 
0.2, 
lOO 
160 200 
Input 
S 
0.8 
0.6 
0.4 
0.2' 
60 100 160 200 
ltrre 
Figure 2: Time dependence of the fraction rn t of output neurons which fire simul- 
taneously compared to the corresponding fraction of input signals s t for n = 20 
and O/or = 0.225. The input variables  are independently distributed according to 
P() = pS(I - 1) + (1 - p)5() with p = 0.1. If more than five input signals with 
 = 1 coincide, the entire population will fire in synchrony two time steps later, 
i.e. rn t+2 = 1. Note the "random" appearance of the interburst intervals. 
network detects coincidences and signals this by a synchronized burst of neuronal 
activity followed by a brief respite of activity (Figure 2). 
The time dependence of m t given by eq. (5) can be written as: 
s t for0<m t< & 
rn + = 1 for 0_ < rn  < 1 (6) 
0 for rn  = 1 
By introducing functions A(m), B(m), C(m) which take on the value 1 in the 
intervals specified for m = rn t in eq. (6), respectively, and zero elsewhere, rn + can 
be written as: 
rn '+ = stA(rnt) + 1. B(m') + O . C(rn') (7) 
This equation can be iterated, yielding an explicit expression for rn t as a function 
of the external inputs st+l,... s o and the initial value m: 
m' = ) o) (8) 
) 
with the matrix 
A(s) 0 I ) 
M(s)= B(,) 0 0 
c(s) o 
Burst Synchronization without Frequency Locking in a Completely Solvable Network Model 121 
Eq. (8) shows that the dynamics of the network can be solved explicitly, by itera- 
tively applying M, t - I number of times to the initial network configuration. 
3 
DISTRIBUTION OF BURSTS AND TIME 
CORRELATIONS 
The synchronous activity at time t depends on the specific realization of the input 
signals at different times (eq. 8). In order to get rid of this ambiguity we resort to 
averaged quantities where averages are understood over the distribution tS{s t} of 
1 r t 
inputs s t =  Y]-i=x i' A very useful averaged quantity is the probability pt(m), 
describing the fraction m of simultaneously firing neurons at time t. pt(m) is related 
to the probability distribution P{s t } via: 
= s0}]) (9) 
where (...) denotes the average with respect to/5{st} and mr{st-X,... s } is given 
by eq. (8). If the input signals  are uncorrelated in time, rn t+ depends according 
to eq. (7) only on rn t, and the time evolution of Pt(m) can be described by the 
Chapman-Kolmogorov equation. We then find: 
Pt(m) = Poo(m) + [P(m) - Poo(m)]. f(t) (10) 
where 
1 
Poo(m) -- 1 q- 2r/[/5(m) q- r/8(m - 1) q- r/8(m)] (11) 
is the limiting distribution which evolves from the initial distribution P(m) for large 
times, because the factor f(t) = r/ cos(gt), where 
decays exponentially with time and rl = fo  tS(s)B(s)ds = foi/, tS(s)ds. Notice that 
0 _< r/_< 1 holds (for more details, see Koch and Schuster, 1992). 
The limiting equilibrium distribution Poo(m) evolves from the initial distribution 
P(m) in an oscillatory fashion, with the building up of two delta-functions at m = 1 
and m = 0 at the expense of P(m). This signals the emergence of synchronous 
bursts, i.e., rn t = 1, which are always followed at the next time-step by zero activity, 
i.e., m t+ = 0 (see also Fig. 2). The equilibrium value for the mean fraction of 
synchronized neurons is 
<m ) - drnPoo(m)m - (s) q- r/ (12) 
l+2r/ 
1 indicating an 
which is larger than the initial value (s) = fo  dsP(s)s, for 
increase in synchronized bursting activity. 
It is interesting to ask what type of time correlations will develop in the output 
of our network if it is stimulated with uncorrelated noise, . The autocovariance 
function is 
C(r) = lim [(mt+m t} - (mr}2], (13) 
can be computed directly since rn t and Poo(m) are known explicitly. We find 
C(r) = 5noCo + (1 - 5,,o)C, rll'l/' cos(fir + g) (14) 
122 Schuster and Koch 
0.? 
0.6 
0.2 
-0.2{ 
-0.5 
-0. '75, 
0.'75. 
0.5- 
O. 25. 
-0.25. 
-0.5. 
-0. '75. 
Figure 3: Time dependence 
values of r/ = fo1/,odsP(s). 
T = 3.09, while the bottom 
T = 3.50. Note the different 
TIn'e 
of the auto-covariance function C(r) for two different 
The top figure corresponds to r/ = 0.8 and a period 
correlation function is for r/ = 0.2 with an associated 
time-scales. 
with ,0 the Kroneker symbol (,0 = 1 for r = 0 and 0 else). Figure 3 shows 
that C(r) consists of two parts. A delta peak at r = 0 which reflects random 
uncorrelated bursting and an oscillatory decaying part which indicates correlations 
in the output. The period of the oscillations, T = 2r/f, varies monotonically 
between 3 < T < 4 as 0/w moves from zero to one. Since r/is given by foI/,o tS(s)ds, 
we see that the strengths of these oscillations increases as the excitatory coupling 
ov increases. The emergence of periodic correlations can be understood in the limit 
0/w --+ 0, where the period T becomes three (and r I = fo  tS(s)ds = 1), because 
according to eq. (t3), m t = 0 is followed by m t+l = s t which leads for 0/w --+ 0 
always to rn t+2 = 1 followed by rn t+3 = 0. In other words, the temporal dynamics 
of m t has the form OsllOs410s?lOslO .... In the opposite case of 0/or --+ 1, r/ 
converges to 0 and the autocovariance function C(r) essentially only contains the 
peak at r = 0. Thus, the output of the network ranges from completely uncorrelated 
0 __+ 0. The power spectrum of 
noise for 0/or m 1 to correlated periodic bursts for  
the system is a broad Lorentzian centered at the oscillation frequency, superimposed 
onto a constant background corresponding to uncorrelated neural activity. 
It is important to discuss in this context the effect of the size n of the network. If the 
input variables ,t. are distributed independently in time and space with probabilities 
15i(), then the distribution P(s) has a width which decreases as 1/v/- as n --+ oo. 
Therefore, in a large system r/: f/ P(s)ds is either 0 if O/w > 
where (s} is the mean value of s, which coincides for n --+ oo with the maximum of 
Burst Synchronization without Frequency Locking in a Completely Solvable Network Model 123 
/5(s). If r/= 0 the correlation function is a constant according to eq. (14), while the 
system will exhibit undamped oscillations with period 3 for r/= 1. Therefore, the 
irregularity of the burst intervals, as shown, for instance, in Fig. 2, is for independent 
,t. a finite size effect. Such synchronized dephasing due to finite size has been 
reported by Sompolinsky el al. (1989). 
However, for biologically realistic correlated inputs ,t., the width of b(s) can remain 
finite for n >> 1. For example, if the inputs ,...,t, can be grouped into q 
correlated sets ...,..., ...,tq ..., with finite q, then the width of/5(s) 
scales like 1/Vrff. Our model, which now effectively corresponds to a situation with 
a finite number q of inputs, leads in this case to irregular bursts which mirror and 
amplify the correlations present in the input signals, with an osci]]atory component 
superimposed due to the dynamical threshold. 
4 CONCLUSIONS AND DISCUSSION 
We here suggest a mechanism for burst synchronization which is based on the fact 
that excitatory coupled neurons fire in synchrony whenever a sufficient number of in- 
put signals coincide. In our model, common inhibition shuts down the activity after 
each burst, making the whole process repeatable, without entraining any signals. It 
is rather satisfactory to us that our simple model shows qualitative similar dynamic 
behavior of the much more detailed biophysical simulations of Bush and Douglas 
(1991). In both models, all-to-all excitatory coupling leads--together with common 
inhibition--to burst synchronization without frequency locking. In our analysis we 
updated all neurons in parallel. The same model has been investigated numerically 
for serial (asynchronous) updating, leading to qualitatively similar results. 
The output of our network develops oscillatory correlations whose range and am- 
plitude increases as the excitatory coupling is strengthened. However, these oscil- 
lations do not depend on the presence of any neuronal oscillators, as in our earlier 
models (e.g., Schuster and Wagner, 1990; Niebur et ai.,1991). The period of the 
oscillations reflects essentially the delay between the inhibitory response and the 
excitatory stimulus and only varies little with the amplitude of the excitatory cou- 
pling and the threshold. The crucial role of inhibitory interneurons in controlling 
the 40 Hz neuronal oscillations has been emphasized by Wilson and Bower (1992) 
in their simulations of olfactory and visual cortex. Our model shows complete syn- 
chronization, in the sense that all neurons fire at the same time. This suggests 
that the occurrence of tightly synchronized firing activity across neurons is more 
important for feature linking and binding than the locking of oscillatory frequencies. 
Since the specific statistics of the input noise is, via coincidence detection, mirrored 
in the burst statistics, we speculate that our network--acting as an amplifier for 
the input noise can play an important role in any mechanism for feature linking 
that exploits common noise correlations of different input signals. 
Acknowledgement s 
We thank R. Douglas for stimulating discussions and for inspiring us to think about 
this problem. Our collaboration was supported by the Stiftung Volkswagenwerk. 
The research of C.K. is supported by the National Science Foundation, the James 
124 Schuster and Koch 
McDonnell Foundation, and the Air Force Office of Scientific Research. 
References 
Bush, P.C. and Douglas, R.J. "Synchronization of bursting action potential dis- 
charge in a model network of neocortical neurons." Neural Computation 3: 19-30, 
1991. 
Crick, F. and Koch, C. 'rowards a neurobiological theory of consciousness." Sem- 
inars Neurosci. 2: 263-275, 1990. 
Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M. and Re- 
itboeck, H.J. "Coherent oscillations: a mechanism of feature linking in the visual 
cortex?" Biol. Cybern. 60: 121-130, 1988. 
Gray, C.M., Engel, A.K., KSnig, P. and Singer, W. "Stimulus-dependent neuronal 
oscillations in cat visual cortex: Receptive field properties and feature dependence." 
Eur. J. Neurosci. 2: 607-619, 1990. 
Gray, C.M., Konig, P., Engel, A.K. and Singer, W. "Oscillatory response in cat 
visual cortex exhibits inter-columnar synchronization which reflects global stimulus 
attributes." Nature 338: 334-337, 1989. 
Horn, D. and Usher, M. "Excitatory-inhibitory networks with dynamical thresh- 
olds." Int. J. of Neural Systems 1: 249-257, 1990. 
Koch, C. and Schuster, H. "A simple network showing burst synchronization with- 
out frequency locking." Neural Computation, in press. 
McCulloch, W.S. and Pitts, W.A. "A logical calculus of the ideas immanent in 
neural nets." Bull. Math. Biophys. 5: 115-137, 1943. 
Niebur, E., Schuster, H.G., Kammen, D.M. and Koch, C. "Oscillator-phase coupling 
for different two-dimensional network connectivities." Phys. Rev. A., in press. 
Schuster, H.G. and Wagner, P. "A model for neuronal oscillations in the visual cor- 
tex: I Mean-field theory and the derivation of the phase equations." Biol. Cybern. 
64: 77-82, 1990. 
Sompolinsky, H., Golomb, D. and Kleinfeld, D. "Global processing of visual stimuli 
in a neural network of coupled oscillators." Proc. Natl. Acad. Sci. USA 87: 
7200-7204, 1989. 
yon der Malsburg, C. and Schneider, W. "A neural cocktail-party processor." Biol. 
Cybern. 54: 29-40, 1986. 
Wilson, M.A. and Bower, J.M. "Cortical oscillations and temporal interactions in 
a computer simulation of piriform cortex." J. Neurophysiol., in press. 
