Stochastic Dynamics of Three-State 
Neural Networks 
Toru Ohira 
Sony Computer Science Laboratory 
3-14-13 Higashi-gotanda, 
Tokyo 141, Japan 
ohira@csl.sony. co.jp 
Jack D. Cowan 
Depts. of Mathematics and Neurology 
University of Chicago 
Chicago, IL 60637 
cowan@synapse.uchicago.edu 
Abstract 
We present here an analysis of the stochastic neurodynamics of 
a neural network composed of three-state neurons described by 
a master equation. An outer-product representation of the mas- 
ter equation is employed. In this representation, an extension of 
the analysis from two to three-state neurons is easily performed. 
We apply this formalism with approximation schemes to a sim- 
ple three-state network and compare the results with Monte Carlo 
simulations. 
I INTRODUCTION 
Studies of single neurons or networks under the influence of noise have been a con- 
tinuing item in neural network modelling. In particular, the analogy with spin 
systems at finite temprature has produced many important results on networks of 
two-state neurons. However, studies of networks of three-state neurons have been 
rather limited (Meunier, Hansel and Verga, 1989). A master equation was intro- 
duced by Cowan (1991) to study stochastic neural networks. The equation uses the 
formalism of "second quantization" for classical many-body systems (Doi, 1976a; 
Grassberger and Scheunert, 1980), and was used to study networks of of two-state 
neurons (Ohira and Cowan, 1993, 1994). In this paper, we reformulate the master 
equation using an outer-product representation of operators and extend our previ- 
ous analysis to networks of three-state neurons. A hierarchy of moment equations 
for such networks is derived and approximation schemes are used to obtain equa- 
2 72 Toru Ohira, Jack D. Cowan 
Figure 1: Transition rates for a three-state neuron. 
tions for the macroscopic activities of model networks. We compare the behavior 
of the solutions of these equations with Monte Carlo simulations. 
2 THE BASIC NEURAL MODEL 
We first introduce the network described by the master equation. In this network 
(Cowan, 1991), neurons at each site, say the ith site, are assumed to cycle through 
three states: "quiescent", "activated" and "refractory", labelled 'qi', 'ai', and 'ri' 
respectively. We consider four transitions: q - a, r - a, a - 7', and r - q. Two of 
these, q - a and r - a, are functions of the neural input current. We assume these 
are smoothly increasing functions of the input current and denoted them by 01, and 
02. The other two transition rates, a - r, and r - q, are defined as constants c and 
/. The resulting stochastic transition scheme is shown in Figure 1. We assume that 
these transition rates depend only on the current state of the network and not on 
past states, and that all neural state transitions are asynchronous. This MarkovJan 
assumption is essential to the master equation description of this model. 
We represent the state of each neuron by three-dimensional basis vcctors using the 
Dirac notation lai >, Iri > and Iqi >. They correspond, in more standard vector 
notation, to: 
Iqi >- 0 , la, >= 0 , Ir, >- 1 (1) 
1 i 0 i 0 
We define the inner product of these states as 
< ailai >=< qilqi >=< rilri >= 1, (2) 
< qi[ai >--< ailqi >----< rilai >=< ailri >=< rilqi >=< qilri >= 0. (3) 
Let the states (or configurations) of a network be represented by {1 >}, the direct 
product space of each neuron in the netxvork. 
Ift >= Iv > Iv2 >... Ivey >, vi = ai, ri, or qi. (4) 
Let P[Ft, t] be the probability of finding the network in a particular state ft at time 
t. We introduce the "neural state vector" for N neurons in a network as 
I(t) >= y P[n, t]l n >, (5) 
{n} 
Stochastic Dynamics of Three-State Neural Networks 2 73 
where the sum is taken over all possible network states. 
With these definitions, we can write the master equation for a network with the 
transition rates shown in Figure 1, using the outer-product representations of op- 
erators (Sakurai, 1985). For exmnple: 
0 0 1) 
la >< ql = 0 0 0 
0 0 0 
(6) 
s=a, r ,q 
For example, 
We note the following relations, 
(ai(t) + ((qi(t) + ((ri(t) = 1 (12) 
d 
The master equation then takes the form of an evolution equation: 
0 
OtiS(t) >= Lib(t) > (7) 
with the network "Liouvillian" L given by: 
N N N 
L - o E(lai)(ai[- ]ri)(ai[)+ E([,'i)(,'i[- [ai)(,'il)O:( E wijlaj)(ajm 
i----1 i--1 j----1 
N 1 
-t-/f=i(lri)(ri[ - Iqi)(r,I) + i=l(Iqi)(qil - lai)(qimOl(w -]f=l wijlaj)(ajl).(s) 
where  is an average number of connections to each neuron, and wij is the "weight" 
from the jth to the ith neuron. Thus the weights are normalized with respect to 
the average number  of connections per neuron. 
The master equation given here is the same as the one introduced by Cowan using 
Gell-Mann matrices (Cowan, 1991). However, we note that with the outer-product 
representation, we can extend the description from two to three-state neurons sim- 
ply by including one more basis vector. 
In analogy with the analysis of two-state neurons, we introduce the state vector: 
N 
( ff q-] = H(qi(qil + ri(ri[ + ai(ail). (9) 
i--1 
where the product is taken as a direct product, and ai, ri, and qi are parameters. 
We also introduce the point moments ((ai(t))), ((qi(t))), and ((ri(t))) as the 
probability that the ith neuron is active, quiescent, and refractory respectively, at 
time t. Similm'ly, we can define the multiple moment, for example, ((aiqjrk... (t))) 
as the probability that the ith neuron is active, the jth neuron is quiescent, the kth 
neuron is refractory and so on at time t. Then, it can be shown that they are given 
by: 
(t)>> = (a = 11s)(sil Isj)(sl 
(10) 
2 74 Toru Ohira, Jack D. Cowan 
3 THE HIERARCHY OF MOMENT EQUATIONS 
We can now obtain an equation of motion for the moments. As is typical in the 
case of many-body problems, we obtain an analogue of the BBGKY hierarchy of 
equations (Doi, 1976b). This can be done by using the definition of moments, the 
master equation, and the a-r-q state vector. We show the hierarchy up to the second 
order: 
0 1N 1N 
Ot <<ai>> = ct<<ai>> -- <<riO2(  Z wijaj )> > _ <<qiO1 (  y wljaj )>> 
j-'t 
0 1 N 
a <<">> = -<<a>> + '<<,.>> + <<,'2 (Y' a)>> 
j=l 
0 I N 
j=l 
(14) 
(15) 
(16) 
0 N 1 N 
k=l k=l 
-<<qiaiO1 ( 1 N 
E=, w.,a,)>> - <<aqO,( ' V.N ,,a,)>> (17) 
o 
-<<rirj>> = --c(<<ria1>> + <<airj>>) + 
I N i N 
+ 
k=l k=l 
(18) 
We note that since 
<<ai>> + <(ri>) + ((qi)) -- 1, (20) 
one of the parameters can be eliminated. We also note that the equations are 
coupled into higher orders in this hieras-chy. This leads to a need for approximation 
schemes which can terminate the hierarchy at an appropriate order. 
In the following, we introduce first and the second moment level approximation 
schemes. For simplicity, we consider the special case in which 0 and 02 are linear 
and equal. 
1 N 1 N 1 N 
(21) 
With the above simplication the first moment (mean field) approximation leads to: 
Stochastic Dynamics of Three-State Neural Networks 275 
o 
-- <<ai>> -- a<<ai>> -- '(<<ri>> + <<qi>>) 
where 
I N 
k-'-I 
We also obtain the second moment approximation as: 
(22) 
(23) 
(24) 
(25) 
0 I N 
j--1 
0 1N 
0 I N 
-- O--<<qi>> = --t3<<ri>> 4-  j__ wij<<qiaj>>, 
(26) 
(27) 
(28) 
0 
-C<aiaj> = 2aCCaiaj>> - wij( CCriaj>> -{- <Cqiaj>> ) 
--wji(CCairj>> 4- 
0 
(29) 
(30) 
0 
--CCairj>> = -a( CCaiaj>> -- <Cairj>> ) 4- ]CCairj>> 4- jiC<airj>> 
--wij( rirj "{- qirj  ), 
(31) 
where 
N 
wtm = -[ E wt<<a:>> + W,m]. (32) 
,=l ,q,. m) 
We note that the first moment dynamics obtained via the first approximation differs 
from that obtained from the second moment approximation. In the next section, 
we briefly examine this difference by comparing these approximations with Monte 
Carlo simulations. 
2 76 Toru Ohira, Jack D. Cowan 
4 COMPARISON WITH SIMULATIONS 
In this section, we compare first and second moment approximations with Monte 
Carlo simulation of a one dimensional ring of three-state neurons. This was studied 
in a previous publication (Ohira and Cowan, 1993) for two-state neurons. As shown 
there, each three-state neuron in the ring interacts with its two neighbors. 
More precisely, the Liouville operator is 
N N 
L = c y(]ai)(ail- ]ri)(ail) + t3 Z(]ri)(ri]- Iqi)(ri]) (33) 
i=1 i=1 
N 
1 
+w2 (Iri)(ril - lal)(ril)(lai+)(ai+xl + lai-1}(ai-lD 
i=1 
N 
1 
+w -.(Iqi)(qil - lai)(qil)(lai+)(ai+ll + lai-1)(ai-xl) 
i=1 
We now define the dynamical variables of interest as follows: 
I I 1 
i--1 i--1 i--1 
1  1  1  
Then, for this network, the first moment approximation is given by 
0 
--Xa ---- Ol -- W2a r -- WlXqXa, 
o 
Xq 
The second moment approximation is given by 
--01X -- )r n t' W2XqXa, 
1 -- Xa -- 
o 
Ot Xa 
o 
o 
-- aa 
o 
(34) 
(35) 
(36) 
(37) 
Stochastic Dynamics of Three-State Neural Networks 277 
Monte Carlo simulations of a ring of 10000 neurons were performed and compared 
with the first and second moment approximation predictions. We fixed the following 
parameters: 
a=l.O, /=0.2, w,=O.Ol.wo, w:=O.6.wo (38) 
(A) (B) 
Figure 2: Comparison of Monte Carlo simulations (dots) with the first moment 
(dashed line) and the second moment (solid line) approximations for the three state 
case with the fraction of total active and refractory state variables 
(B). Each graph is labeled by the values of 
We varied w0 and sampled the numerical dynamics of these parameters. Some 
comparisons are shown in Figure 2 for the time dependence of the total number 
of active and refractory state variables. We clearly see the improvement of the 
second over the first moment level approximation. More simulations with different 
parameter ranges remain to be explored. 
5 CONCLUSION 
We have introduced here a neural network master equation using the outer-product 
representation. In this representation, the extension from two to three-state neu- 
rons is transparent. We have taken advantage of this natural extension to analyse 
three-state networks. Even though the calculations involved are more intricate, we 
2 78 Toru Ohira, Jack D. Cowan 
have obtained results indicating that the second moment level approximation is sig- 
nificantly more accurate than the first moment level approximation. We also note 
that as in the two-state case, the first moment level approximation produces more 
activation than the simulation. Further analytical asd theoretical investigations 
are needed to fully uncover the dynamics of three-state networks described by the 
master equation introduced above. 
Acknowledgements 
This work was supported in part by the Robert R. McCormick fellowship at the 
University of Chicago, and in part by grant No. N0014-89-J-1099 from the US 
Department of the Navy, Office of Naval Research. 
References 
Cowan JD (1991) Stochastic neurodynamics in Advances in Neural Information 
Processing Systems (D. S. Touretzky, R. P. Lippman, J. E. Moody, ed.), vol. 3, 
Morgan Kaufmann Publishers, San Mateo 
Doi M (1976a) Second quantization representation for classical many-particle sys- 
tem. J. Phys. A: Math Gem 9:1465-1477. 
Doi M (1976b) Stochastic theory of diffusion-controlled reactions. J. Phys. A: Math. 
Gen. 9:1479. 
Grassberger P, Scheunert M (1980) Fock-space methods for identical classical ob- 
jects. Fortschritte der Physik 28:547 
Meunier C, Hansel D, Verga A (1989) Information processing in three-state neural 
networks. J. Stat. Phys. 55:859 
Ohira T, Cowan JD (1993) Master-equation approach to stochastic neurodynamics. 
Phys. Rev. E 48:2259 
Ohira T, Cowan JD (1994) Feynman Diagrams for Stochastic Neurodynamics. In 
Proceedings of Fifth Australian Conference of Neural Networks, pp218-221 
Sakurai JJ (1985) Modern Quantum Mechanics. Benjamin/Cummings, Menlo Park 
