192 
PHASE TRANSITIONS IN NEURAL NETWORKS 
Joshua Chover 
University o Wisconsin, Madison, WI 53706 
ABSTRACT
Various simulations o cortical subnetworks have evidenced 
something like phase transitions with respect to key parameters. 
We demonstrate that. such transitions must. indeed exist_ in analogous 
ininite array models. For related inite array models classical 
phase transit.ions (which describe steady-state behavior) may not. 
exist., but. there can be distinct. qualitative changes in 
("metastable") transient. behavior as key system parameters pass 
through critical values. 
INTRODUCTION 
Suppose that. one st.imulates a neural network - actual or 
simulated - and in some manner records the subsequent iring 
activity o cells. Suppose urther that. one repeats the experiment. 
or dierent values o some parameter (p) o the system: and that. 
one inds a "critical value" (pc) o the parameter, such that. 
(say) or values P ) Pc the activity tends to be much higher than 
it. is or values P < Pc' Then, by analogy with statistical 
mechanics (where, e.g., p may be temperature, with critical 
values or boiling and reezing) one can say that. the neural 
network undergoes a "phase transition" at. Pc' Intracellular phase 
transitions, parametrized by membrane potential, are well known. 
Here we consider intercellular phase transitions. These have been 
evidenced in several detailed cortical simulations: e.g., of the 
pitiform cortex 1 and of the hippocampus 2 In the pitiform case, 
the parameter p represented the frequency of high amplitude 
spontaneous EPSPs received by a typical pyramidal cell; in the 
hippocampal case, the parameter was the ratio of inhibitory to 
excitatory cells in the system. 
By what. mechanisms could approach to, and retreat. from, a 
critical value of some parameter be brought about? An intriguing 
conjecture is that. neuromodulators can play such a role in certain 
3 
networks; temporarily raising or depressing synaptic efficacies 
What. possible interesting consequences could approach to 
criticality have for system performance. Good effects could be 
these: for a network with plasticity, heightened firing response 
to a stimulus can mean faster changes in synaptic eficacies, which 
would bring about_ faster memory storage. More and longer activity 
could also mean faster access to memory. A bad effect. o 
@ American Institute of Physics 1988 
193 
near-criticality - depending on other parameters - can be wild, 
epileptiorm activity. 
Phase transitions as they might. relate to neural networks have 
4 
been studied by many authors Here, or clarity, we look at. a 
particular category o network models - abstracted rom the 
piriorm cortex setting reerred to above - and show the ollowing: 
a) For "elementary" reasons, phase transition would have to 
exist i there were ininitely many cells; and the near-subcritical 
state involves prolonged cellular iring activity in response to an 
initial stimulation. 
b) Such prolonged iring activity takes place or analogous 
large inite cellular arrays - as evidenced also by computer 
simulations. 
What. we shall be examining is space-time patterns which 
describe the mid-term transient. activity o (Markovian) systems 
that. tend to silence (with high probability) in the long run. 
(There is no reerence to energy unctions, nor to long-run stable 
iring rates - as such rates would be zero in most. o our cases.) 
In the ollowing models time will proceed in discrete steps. 
(In the more complicated settings these will be short. in comparison 
to other time constants, so that_ the eect o quantization becomes 
smaller.) The parameter p will be the probability that at. any 
given time a given cell will experience a certain amount. o 
excitatory "spontaneous iring" input.: by itsel this amount. will 
be insuicient. to cause the cell to ire, but. in conjunction with 
suiciently many excitatory inputs rom other cells it. can assist. 
in reaching iring threshold. (Other related parameters such as 
average iring threshold value and average eficcy value give 
similar results.) In all the models there is a refractory period 
after a cell fires, during which it cannot fire again; and there 
may be local (shunt. type) inhibition by a firing cell on near 
neighbors as well as on itself - but. there is no long-distance 
inhibition. We look first. at. limiting cases where there are 
infinitely many cells and - classically - phase transition appears 
in a sharp orm. 
A "SIMPLE" MODEL 
We consider an infinite linear array of similar cells which 
obey the following rules, pictured in Fig. 1A: 
(i) If cell k fires at. time n, then it. must. be silent. 
at. time n+l; 
(ii) if cell 
neighbors k-1 and 
at. time n+l; 
(iii) if cell k is silent at time n and Just one of its 
neighbors (k-1 or k+l) fires at. time n, then cell k will 
fire at time n+l with probability p and not. fire with 
probability l-p, independently of similar decisions at. other 
cells and at. other times. 
k is silent. at. time n but. both of its 
k+l do fire at. time n, then cell k fires 
194 
Fig. 1. "Simple model". A: firing rules; cells are represented 
horizontally, time proceeds downwards; filled squares 
denote firing. B: sample development. 
Thus, efecttvely, signal propagation speed here is one cell 
per unit. time, and a cell's firing threshold value is 2 (EPSP 
units). I{ we stimulate one cell to {ire at time n--O, will its 
influence necessarily die out or can it. go on forever? (See 
Fig. lB.) For an answer we note that. in this simple case the 
firing pattern (if any) at. time n must. be an alternating stretch 
o{ firing/silent cells o{ some length, call it. L . Moreover, 
n 
2 
Ln+ 1 = L +2 with probability p (when there are sponteneous 
n 
firing assists on both ends o{ the stretch), or Ln+ 1 = Ln-2 with 
probability (l-p) 2 (when there is no assist at. either. end o{ the 
stretch), or Ln+ 1 = L n with probability 2p(1-p) (when there is 
an assist. at. just. one end o{ the stretch). 
Starting with any finite alternating stretch L O, the 
successive values L constitute a "random walk" among the 
n 
nonnegative integers. Intuition and simple analysis 5 lead to the 
same conclusion: i{ the probability {or L n to decrease ((l-p) 2) 
is greater than that_ {or it. to increase (p2) _ i.e. if the average 
step taken by the random walk is negative - then ultimately L 
n 
will reach 0 and the firing response dies out. Contrariwise, i{ 
195 
2 l_p)2 
p > ( then the L can drift_ to even higher values with 
n 
positive probability. In Fig. 2A we sketch the probability for 
ultimate die-out as a function of p; and in Fig. 2B, the average 
time until die out. Figs. 2A and B show a classic example of phase 
transition (Pc = 1/2) for this infinite array. 
Fig. 2. Critical behavior. A: probability of ultimate die out. (or 
of reaching other traps, in finite array case). 
B: average time until die-out (or for reaching other 
traps). Solid curves refer to an infinite array; dashed, 
to finite arrays. 
MORE COMPI.EX MODELS 
For an infinite linear array of cells, as sketched in Fig. 3 , 
we describe now a much more general (and hopefully more realistic) 
set. of rules: 
(i') A cell cannot. fire, nor receive excitatory inputs, at. 
time n if it has fired at any time during the preceding m R time 
units (refraction and feedback inhibition). 
(ii') Each cell x has a local "inhibitory neighborhood" 
consisting of a number (j) of cells to its immediate right. and 
left_, The given cell x cannot. fire or receive excitatory inputs 
at time n if any other cell y in its inhibitory neighborhood 
has fired at_any time between t and t+m I units preceding n, 
where t is the time it. would take for a message to travel from y 
to x at. a speed of v I cells per unit time. (This rule 
represents local shuntstype inhibition.) 
(iii') Each cell x has an "excitatory neighborhood" 
consisting of a number (e) of cells to the immediate right_and left 
of its inhibitory neighborhood. If a cell y in that. neighborhood 
fires at. a certain time, that firing causes a unit impulse to 
travel to cell x at a speed of v E cells per unit. time. The 
impulse is received at. x subject to rules (i') and (ii'). 
196 
(iv') All cells share a "iring threshold" value 0 and an 
"integration time constant? s (s (0). In addition each cell, at. 
each time n and independently o other times and other cells, can 
receive a random amount. X o "spontaneous excitatory input?. 
n 
The variable X can have a general distribution; however, or 
n 
simplicity we suppose here that. it. assumes only one o two values: 
b or O, with probabilities p and 1-p respectively. (We 
suppose that. b ('0, so that. the spontaneous "assist? itsel is 
insuicient. or iring.) The above quantities enter into the 
ollowing firin rule: a cell will fire at. time n if it_ is not. 
prevented by rules (i') and (ii') and i the total number of inputs 
from other cells, received during the integration "window" lasting 
between times n-s+l and n inclusive, plus the assist. X n, 
equals or exceeds the threshold O. 
(The propagation speeds v I and V E and the neighborhoods 
are here given leftsright symmetry merely or ease in exposition.) 
Fig. 3. Hessage travel in complex model: 
(i')-(iv'). 
see text. rules 
Will such a model display phase transition at. some critical 
value of the spontaneous firir frequency p ? The dependence of 
responses upon the initial conditions and upon the various 
parameters is intricate and will afect the answer. We briefly 
discuss here conditions under which the answer is again yes. 
(1) For a given configuration of parameters and a given 
initial stimulation (of a stretch of cont.iuous cells) we compare 
the development. of the model's firing response first. to that. of an 
auxiliary "more active" system: Suppose that. L now denotes the 
n 
distance at. time n between the left and right-most cells which 
are either firing or in refractory mode. Because no cell can fire 
without_ influence rom others and because such influence travels at_ 
a given speed, there is a maximal amount_ (D) whereby L can 
n+l 
exceed L n. There is also a maximum probability Q(p) - which 
197 
depends on the spontaneous firing parameter p - that. Ln+ 1  L n 
(whatever n). 'We can compare L with a random walk "A" 
n n 
defined so that_ An+ 1 = An+D with prohability Q(p) and 
 is 
An+ 1 = An-1 with probability 1-Q(p) At each transition, A n 
more likely to increase than L n. Hence L n is more likely to die 
out. than A n . In the many cases where Q(p) tends to zero as p 
does, the average step size of A n (viz., DQ(p)+(-1)(1-Q(b))) 
will become negative for p below a "critical" value Pa' Thus, 
as in the "simple" model above, the probability of ultimate die-out 
for the A n , hence also for the L of the complex model will be 
n ' 
1 when 0  p < Pa' 
(2) There will be a phase transition for the complex model if 
its probability of die out.- given the same parameters and initial 
stimulation is in (1) - becomes less than 1 for some p values 
with Pa < p < 1. Comparison of the complex process with a simpler 
"less active" process is difficult. in general. However, there are 
parameter configurations which ultimately can channel all or part. 
of the firing activity into a (space-time) sublattice analgous to 
that_ in Fig. 1. Fig. 4 illustrates such a case. For p 
sufficiently large there is positive probability that. the activity 
will not. die out, just as in the "simple" model. 
Fig. 4. Activity on a sublattice. (Parameter values: J=2, e=6, 
MR=2, Mi=I, VR=Vi=I, 0--3, s=2, and b=l.) Rectangular 
areas indicate refraction/inhibition; diagonal lines, 
excitatory influence. 
198 
LARGE FINITE ARRAYS 
Consider now a large finite array of N cells, again as 
sketched in Fig. 3 ; and operating according to rules similar to 
{i'}-{iv'} above, with suitable modifications near the edges. 
Appropriately encoded, its activity can be described by a (huge} 
Markov transition matrix, and - depending on the initial 
stimulation - must. tend 5 to one of a set. of steady-state 
distributions over firing patterns. For example, (a) if N is 
odd and the rules are those for Fig. 1, then extinction is the 
unique steady state, for any p < 1 (since the L form a random 
n 
walk with "reflecting" upper barrier}. But, (} if N is even 
and the cells are arranged in a ring, then, for any p with 
0 < p < 1, both extinction and an alternate flip-flop firing 
pattern of period 2 are "traps" for the system - with relative long 
run probabilities determined by the initial state. See the dashed 
line in Fig. 2A for the extinction probability in the () case, 
and in Fig. 2B for the expected time until hitting a trap in the 
(a) case (p<) and the () case. 
What qualitative properties related to phase transition and 
critical p values carry over from the infinite to the finite 
array case? The (a) example above shows that lon term activity 
may now be the same for all 0 < p < I but_ that parameter 
intervals can exist. whose key feature is a particularly large 
expected time before the system hits a trap. (Again, the critical 
region can depend upon the initial stimulation.) Prior to being 
trapped the system spends its time among many states in a kind of 
"metastable" equilibrium. (We have some preliminary theoretical 
results on this conditional equilibrium and on its relation to the 
infinite array case. See also Ref. 6 concerning time scales for 
which certain corresponding infinite and finite stochastic automata 
systems display similar behavior.) 
Simulation of models satisfying rules (i')-(iv') does indeed 
display large changes in length of firing activity corresponding to 
parameter changes near a critical value. See Fig. 5 for a typical 
example: As a function of p, the expected time until the system 
is trapped (for the given parameters) rises approximately linearly 
in the interval .05<p<.12, with most. runs resulting in extinction 
- as is the case in Fig. 5A at. time n=115 (for p=.10). But. for 
p>.15 a relatively rigid patterning sets in which leads with high 
probability to very long runs or to traps other than extinction - 
as is the case in Fig. 5B (p=.20) where the run is arbitrarity 
truncated at. n--525. (The patterning is highly influenced by the 
large size of the excitatory neighborhoods.) 
199 
A 
Fig. 5. Space time firing patterns for one configuration of basic 
parameters. (There are 200 cells; j=2, e=178, MR=10, 
Mi=9, VR=Vi=7, =25, s=2, and b=12; 50 are stimulated 
initially.) A: p=.10. B: p=.20. 
CONCLUSION 
Mechanisms such as neuromodulators, which can (temporarily) 
bring spontaneous iring levels - or synapt.ic eicacies, or 
average iring thresholds, or other similar parameters - to 
near-critical values, can thereby induce large ampliication o 
response activity to selected stimuli. The repertoire o such 
responses is an important. aspect_ o the system's unction. 
200 
Acknowledgement: Thanks to C. Bezuidenhout and J. Kane for help 
with simulations.
References
M. Wilson, J. Bower, J. Chover, L. Haberly, 16th Neurosci. 
Soc. Mtg. Abstr. 370.11 (1986). 
R. D. Traub, R. Hiles, R.K.S. Wong, 16th Neurosci. Soc. Htg. 
Abstr. 196.12 (1986). 
A. Selverston, this conference, also, Hodel Neural Networks 
and Behavior, Plenum (1985); E. Harder, S. Hooper, J. Eisen, 
Synaptic Function, Wiley (1987) p.305. 
E.g.: W. Kinzel, Z. Phys. B55, p. 231 (1985); A. Noest. 
Phys Rev. Let_. 57(1), p. 90 (1986); R. Durrett (to appear); 
C. Carpenter, J. Di. Eqns. 23, p.335 (1977); C. Ermentraut, 
S. Cohen, Biol. Cyb. 34, p.137 (1979); H. Wilson, S. Cowan, 
Biophys. J. 12 (1972). 
W. Feller, An Introd. to Prob. Th'y. and Appl'ns. I. Wiley 
(1S) Ch. 14, 15. 
T. Cox and A. Craven (to appear). 
