II It 
Silicon Auditory Processors 
as 
Computer Peripherals 
John Lazzaro, John X, Vawrzynek 
CS Division 
UC Berkeley 
Evans !!all 
Berkeley, C,A 94720 
Zazzarocs. berkeZey. edu, j ohnwcs. berkeZey. edu 
M. Maimwahl*, Massimo Sivilotti t, Dave Gillespie t 
California Institute of'l%chnology 
Pasadena, CA 91125 
Abstract 
Several research group are inl)hmcnl. ing alalog integrated circuit 
models of biological auditory processing. The outl)uts of these 
circuit nodels have taken several k)nns, including vicico format 
for monitor display, simple scanned out. put for oscilloscope display 
and parallel analog outl3uts suitable for da. ta-acquisition systems. 
In this paper, we describe an alternative output method for silicon 
auditory models, suitable for direct interfa. ce to digital computers. 
* Present address: hi. Mallowaid, MI{.(-', Anatomical Neurol)hamacology 
Unit, Mansfield Rd, Oxford OX 1 3TII England. mamvax. oxford. ac .uk 
} Present. address: N!ass Sivilol.ti, 'l'ancr 1/,esca.rch, 180 North Vinedo 
Avenue, Pasadena, CA 911()7, masstanner.com 
+ Present address: Dave Gilh,spic, Sval)l.ics, 2698 ()rcla. rd Parkway, San 
Jose CA, 95134. davegsynaptics. corn 
820 
Silicon Auditory Processors as Computer Peripherals 821 
1. INTRODUCTION 
Several researchers have implemented computational models of biological auditory 
processing, with the goal of incorporating these models into a speech recognition 
system (for a recent review, see (Jankowski, 1992)). These projects have shown the 
promise of the biological approach, sometimes shmving clear performance advan- 
tages over traditional methods. 
The application of these COml)utftional models is limited by their large compu- 
tation and communication req,irement:3, Analog VLSI implementations of these 
neural models may relieve this co,putat. ional burden; several VLSI research groups 
have efforts in this area, and worki,g integrated circuit models of ninny popular 
representations presently cxist. A review of those models is presented in (Lazzaro, 
1991). In this paper, we present an interface method (Mahowald, 1992; Sivilotti, 
1991) that addresses the commtmications iss,es between analog VLSI auditory im- 
plementations and digital processors. 
2. COMMUNICATIONS IN NEURAL SYSTEMS 
Biological neurons con,municate long distances using a pulse representation. Com- 
munications engineers have developed several schemes for communicating on a wire 
using pulses as a[omic units. In these schemes, naximally using the communica- 
tions bandwidth of a wire implies tho mean rate of pulses on the wire is a significan 
fraction of the maxinmm pulse rat,e allowed on the wire. 
Using this criterion, neural systcns use wires wn-y imtficiently. In nost parts of 
brain, most of the wires are esse,tia. lly inac[ive nos[ of lie time. If neural systems 
are not organized to Fully utilize tim available bandwidth of each wire, what does 
neural communication ol)t,inizc? Evidccc suggests that energy conservation is an 
important issue for nmral syst,ens. A sinph' slrat.cgy for energy conservation is 
the reduction of the total nnbcr oF l>lscs i the representat. iot. Many possible 
coding strategies satist, this ewrgy requiretent. 
The strategies observcl in mural svst.ms slyare anotler cotmo prolmrt.y. Neural 
systems o['t.e i,nphmmnt. a class of col)ut. ations i a tanner that produces an 
energy-efficient output encoding as an a{l{litional byproduct. The energy-efficient 
coding is not performed simply For communicat, ion and imnediately reversed upon 
receipt, but is an integral part o[' the new represcntatiot, In this way, energy- 
efficient neural coding is intrinsically (liWercd. h'on engineering data compression 
tech ni ques. 
Temporal adap[ation, lateral inhilfition, a{I spike correlations are examples of neu- 
ral processing methods float per[brm itercsting conputations while producing an 
energy-efficient OUtlint code. These representational principles are [he foundation 
of the neural coniputation and communication method we advocate in this paper. 
In this method, the output units of a. clil) are spiking neuron circuits that use 
energy-efficient coding ttetlols. To commicate t. lfis code off a clip, we use a 
distinctly non-biological aplroach. 
822 Lazzaro, Wawrzynek, Mahowald, Sivilotti, and Gillespie 
3. THE EVENT-ADDRESS PROTOCOL 
The unique characteristics of energy-efficient codes define the remaining off-chip 
communications problem. In the spiking he. ton protocol, the height and width of 
the spike carries no information; the neuron imparts new information only at the 
moment a spike begins. This moment occurs asynchronously; there is no global clock 
synchrouizing the output units. One way of completely specifying the information 
in the output units is an event list,, a tabulation of the precise time each output 
unit begins a new spike. % can use this specification as a basis for an off-chip 
communications systen, that sends an event-list message off-chip at, the moment an 
outpnt neuron begins a new spike. An ewnt-list message includes the identification 
of the output unit,, and the time of firing. A performance analysis of this protocol 
can be found in (Lazzaro et al., 1992). 
Note that an explicit timest. amp for each e,try in the event list is not. necessary, if 
communication latency between the sending chip and the receiver is a constant. In 
this case, the sender si,nply co,nmu,icates, upo, onset of a spike fi'om an output, the 
identity of the output mit; the receiver can appe, d a locally generated timestamp 
to complete the event. If simplified in this man,mr, we refer to the event-list protocol 
as the event-address protocol. 
We have designed a working systen that computes a model of auditory nerve re- 
sponse, in real time, using analog VLSI processing. This system takes as input 
an analog sound source, a(I uses the event-list representation to comnunicate the 
model output to the host computer. 
Board Architecture 
Chip Architecture 
Analog 
Process ng 
Spiking 
Output 
A tray 
Arbiters 
and 
Parallel 
Encoding 
Timer 
Sound hiptit 
I]0 0000000000 
Figure 1. System !)lock diagram, silowing chip architecture, board architectnre, 
and the host. conaputer (S.n IPC). 
Silicon Auditory Processors as Computer Peripherals 823 
4. SYSTEMS IMPLEMENTATION 
Figure 1 is a block diagram of this system. A single VLSI chip computes the auditory 
model response; an array of spiking neuron circuits is the final representation of the 
model. This chip also implements the event-address protocol, using asynchronous 
arbitration circuits. The chip produces a parallel binary encoding of the model 
output, as an asynchronous stream of event addresses. These on-chip operations 
are shown inside the dashed rectangle in Figure 1, labelled Chip Architecture. 
Additional digital processing completes the custom hardware in the system. This 
hardware transforms the event-address protocol into an event-list protocol, by 
adding a time lnarker for each event (16 bit time markers with 20ps resolution). 
In addition, the hardware implements the bus interface to the host computer, in 
conjunction with a commercial interface board. The commercial interface board 
supports 10 MBytes/second asynchronous data transfers between our custom hard- 
ware and the host computer, and includes 8 KBytcs of data. buffers. Our display 
software produces a real-time graphical display of the auditory model response, 
using the X window system. 
5. VLSI CIRCUIT DETAILS 
Figure 2 shows a block diagram of the chip. The analog input signal connects 
to circuits that perform analog processing, Illat are fully described and referenced 
in (Lazzaro et al., 1993). The output of this analog processing is represented by 
150 spiking neurons, arranged in a 30 by 5 array. These are the output units of 
the chip; the event-address protocol communicates the activity of these units off 
chip. At the onset of a spike from an output unit, the array position of the spiking 
unit, encoded as a binary number, appears on the output bus. The asynchronous 
output bus is shown in Figure 2 as the data signals marked Encoded X Output 
(column position) and Encoded Y Output (row position), and the acknowledge 
and request control signals A and R. 
We implemented the event-address protocol as an asynchronous arbitration protocol 
in two dimensions. In this sclenc, an output uil, can access two request lines, 
one associated with its row and one associated with its column. Using a wire-OR 
signalling protocol, any outi)tt. unit on a particular row or colmnn may assert the 
request line. Each request line is paired with an acknowledge line, driven by the 
arbitration circuitry outside the array. Row and columu wires for acknowledge and 
request are explicitly shown il Figure 2, as the lines that form a grid inside the 
output unit array. 
At the onset of a spike, an outprat unit asserts its row request line, and waits for 
a reply on its row a.cktowledge liltc. An a.synchronous arbitratiou system, marked 
in Figure 2 as Y Arbitration Tree, assures only one output row is acknowledged. 
After row acknowledgement, tile output unit asserts its column request line, and 
waits for a reply on its column acknowledge line. The arbitration system is shown 
in detail in Figure 2; four two-input arbiter circuits, shown as rectangles marked 
with the letter A, are connected as a tree t.o arbitrate among the 5 column inputs. 
824 Lazzaro, Wawrzynek, Mahowald, Sivilotti, and Gillespie 
I?, A 
,,ncoded Y Axis 
Sound Input 
Contr(l 
A A 
A 
Encoded X Output 
A 
Figm'c 2. I:;Iock (liagrat )F linc chip. ,qoc icxt for (Iotails. 
Silicon Auditory Processors as Computer Peripherals 825 
R o R._, .4 o ,4  
.4 '2 
:\ To Array 
,,\ To Tree 
(b) 
(c) 
Figm'e 3. Diag,'a,ns of co,,,uicalicm ('i,'c,ils i, tlw ('lip. (a) Two-inpul. 
circuit. (b) Co,drol logic to inl.r{':c,. a,'l)i,'aion logic and o,tl),,t. unil array. 
Outlint unit. ci,'clit.. 
826 Lazzaro, Wawrzynek, Mahowald, Sivilotti, and Gillespie 
Upon the arrival el' both row and column acknowledgements, the output unit re- 
leases boh row and column request lines. Static la. tches, shown in Figm'e 2 as the 
rectangles marked Control Logic, ret, aiu the state of the row and colunto request 
lines. 
Binary encoders transt'orm the row and column acknowledge lines into the output 
data bus. Another column encoder senses tle acknowledgement of any column, and 
sserts the bus coutrol oul, pt R. When tim external device has secured t.le data, 
it responds by a.sserting tlw A signal. The .4 signal clears the static latches in 
the Control Logic blocks ad resets tt. When A is reset,, the data. transfer is 
complete, and the chip is ready For the next communication event. 
Figure 3 shows the details oF the communicalions circuits oF Figure 2. Figure 3(a) 
shows the two-input arbiter circuit used to create the binary arbitration trees in 
Figure 3. This digital circuit takes as input two request signals, Ri and R, and 
produces the associated acknowledge signals A1 and A2. The acknowledgement of 
a request precludes the acknowledgement of a second request. The circuit asserts 
an acknowledge sigal until its associaled r(,q(st is released. 
Re is an auxiliary output signal indicating oitlwr Rl or R.2 has beeu asserted; Ao is 
an auxiliary input sigal l.lal cnal)les the .-I an(I A., out. i)uts. Tim auxiliary signals 
alloxv the two-input arbiler to finctio as an elopement in arl)itration t es, as 
 e shown 
in Figure 2; the Ro and .4o signals o one lew'l oF arbitration couuec[ to [he R and 
A& signals at, t,le next level o[arbit. ration. In two-input operations, the o and Ao 
signals are connected togel. her, as showu in tlw root. arbiter in Figure 2. 
Figure 3(b) shows t. lm circuit inplmentatio o[' the Control Logic blocks i Fig- 
ure 2; this circui[ is rcpeatc{I for each row and colmnn connection. This circuit 
interlaces the output bs control input A. with the arbitration circuitry. If ou[put 
communication is nol in progress, A is el, ground, and A is at Vdd. 
The PFET transistor tnarked as Load acts as a static pullup to the array request 
line (R); output units pull this li}e low to a.sserL a request. The NOR gate inverts the 
array request line, and routes it to the arl)itration tree. When a pending request 
is acknowledged by the tree acknowledge line, the two NFET transistors act to 
latch the array reqttest line. The assertion of A releases the array request line 
and disables the arbitration Lroe rc(lucs[ inpt.; these actions rese[ all sta[e in the 
communications sysLcm. When A is released, [he sys[em is ready to communicate 
a 11ev orelit. 
Figure 3(c) shovs the circuit implementation o['a unit in the output array. In this 
implementation, each output uit is a two-stage low-power axon circuit (Lazzaro, 
199'2). The first axehal stage receives the cecillear input; this axon stage is not 
shown in Figure 3(c). The first st. age couples it.o tle second stage, shown iu Figure 
3(c), via tlc ,_9 and F wires. 
To understancl the operation o1' lhis circuit.. w(, ('olsi(ler l lse transmissiol of a single 
spil:.e. Initially, we associate th(' rel('st lira's It'.,. and l't', are held high hy t.l static 
pullup PFET transistor.,4 slown in !"ignore 3(h); i addition, we assume the a. cktowl- 
edge lines A.,. a.d A, are at gro{I, a(I t.lw oniverting }mf[r input voltage is at 
Silicon Auditory Processors as Computer Peripherals 827 
grOtl I] d. 
When t, he first axona.l stage fires, t, hc S signal changes fi'om ground potent, ial to 
Vdd. At this point the !)ufFer inpt voltage begins t.o increase, a.t a rate determined 
by the analog control voltage P. When t. lte switching threshold oF t. he buffer is 
reached, thc buffer out, l)tt x'olt. agc F swings t.o k4; capacitive fcedback ensures a 
reliable switching transition. At this point, the output unit pulls the request line 
Rv Ion,, and the communical. ions SC(lUece I)cgins. 
Tile Y arbitrat, ion logic replies to lhe /? request 1)y asserting t. he A v line. When 
bot, h F and Ay are asserted, the output, unit pulls the request line R. low. The X 
arbitration logic replies to the t?.. request by asserting the A line. Tile assertion of 
both A and .4y resets the 1)uflr input voltage l,o ground. As a result, the F line 
swings t,o ground potential, the output unit. releases the R and R s lines, and t, he 
first axon stage is enabled. At, tiffs point, the latch circuit, of Figure 3(b) maint, ains 
the state of the R and Ii'.: lines, tmtil it. is cleared by the off-chip acknowledge 
signal. 
Acknowledgements 
Research and protot, ypilg of t, he evcnt-addres., itel'kt. ce took place in Carver Mead's 
laboratory at, Calt, ech: we arc gratefi[ for Iis insighls, cncouragmnet, and support,. 
The Calt, ech-based roseatoll was funded by l. lw ONR, liP, an(1 the Syst. ems Develop- 
ment Foundation. Research anti irol,oiyping of l. he auditory-nerve demonstration 
chip and system tool< place al U(' Berkeley, anl was funded by t, lte NSF (PYI award 
MIPS-895-8568), AT&T, and the ONR (!1{!-N00014-92-.1-1672). 
R,eikrences 
Jankowski, C. R. (1992). "A ('olll)arison of At(lil,ory N'!odcls For Aut, omatic Speech 
Recognition," S.B. Tlwsis, NI IT !)epl, of l.][ccl rical l;htgilmcring and Corn puter Sci- 
ellce. 
Lazzaro, J.P. (1991). "Biological[v-basel audilory signal processing in analog 
VLSI," IEEE A.s'ilomar ('oJroc on ,%gnal.s, NJ.stcn.s. and COmlntlers. 
Lazzaro, J.P. (1992). "l,ow-powcr si[ico Sliking .lrons and axons," IEEE l. ter- 
national Syntposium on C'ircutl.s and ,g]I,slcIH.S, Sail 1)i%o, CA, p. 2220-222't. 
Lazzaro, J., Wawrzynek, J., Nlaltowald, 1., Sivilot. ti, N'I., and Gillespie, D. (1993). 
"Silicon auditory processors as co}ltcr I)eril)herals," IEEE 2a,sactious of Ne'ural 
Nehvorks, May (in press). 
Mahowald, M. (1992). Ph.l). Thesis, Comltation and Neural Systems, California 
Ins[ittte of Tecltology. 
Sivilotti, NI. (1991). "Wirig cosidcral.ions i analog VLSI systems, with appli- 
cations to ficld-progratntalh, networks," (k)llptl,er .qcicncc Technical lieport (Ph. 
D. Thesis), Calilbria lnsil.t.c of qclology. 
