Asynchronous computing

Figure 1.

Figure 1. A simple system of two differential equations (∂x/∂t = -αx + (x-y)(1-x) + αI, ∂y/∂t = -αy + (y-x)(1-y) + αI, I=1, α=0.5, 0 ≤ x,y ≤ 1) has been numerically integrated for 15 seconds using a time step of 25ms. Starting from the initial position (0,0), the blue curve shows the (unique) trajectory of (x,y) for synchronous evaluation while red ones show sample trajectories for asynchronous evaluation). Sources.

Most computational paradigms linked to artificial neural networks (using rate code) or cellular automata use implicitly what is called synchronous evaluation of activity. This means that information at time t+dt is evaluated exclusively on information available at time t. The explicit numerical procedure to perform such a synchronized update is to implement a temporary buffer at the unit level where activity computed at time t+∆t is stored. Once all units have evaluated their activity at time t+∆t, The current activity is replaced by the content of the buffer. We point out that other update procedures have been developed [Lambert:1991] but the basic idea remains the same, namely not to mix information between time t and time t+∆t. To perform such a synchronization, there is thus a need for a global signal that basically tell units that evaluation is over and they can replace their previous activity with the newly computed one. At the computational level, this synchronization is rather expensive and is mostly justified by the difficulty of handling asynchronous models.

For example, cellular automata have been extensively studied during the past decades for the synchronous case and many theorems has been proved in this context. However, some recent works on asynchronous cellular automata showed that the behavior of these same models and associated properties may be of a radical different nature depending on the level of synchrony of the model (you can asynchronously evaluate only a subpart of all the available automata). In the framework of computational neuroscience we may then wonder what is the relevance of synchronous evaluation since most generally, the system of equations is supposed to give account of a population of neurons that have no reason to be synchronized (if they are not provided with an explicit synchronization signal).

In [Taouali:2009] , [Rougier:2010] and [Taouali:2011], we've been studying the effect of asynchronous computation (uniform or non-uniform) on neural networks and more specifically for the case of dynamic neural fields.