Approximate any nonlinear system with spiking artificial neural networks: no training required
Nonlinear computations in spiking neural networks through multiplicative synapses
Artificial (spiking) neural networks (ANNs) have become an important tool in the modelling of biological neuronal circuits. However, they come with caveats: their typical training can be laborious, and after it is done, the complexity of the connectivity obtained can be almost as daunting as the original biological systems we are trying to model.
In this work , Nardin and colleagues summarize and expand upon the Spike Coding Network (SCN) framework , which originally provides a direct method to derive the connectivity of a spiking ANN representing any given linear system. They generalize this framework to approximate any (non-linear) dynamical system, by yielding the connectivity necessary to represent its polynomial expansion. This is achieved by including multiplicative synapses in their network connections. They show that higher polynomial orders can be efficiently represented with hierarchical network structures. The resulting networks not only enjoy many of the desirable features of traditional ANNs, like robustness to (artificial) cell death and realistic patterns of activity, but also a much more interpretable connectivity. This is promptly leveraged to derive how densely connected a neural network of this type needs to be to be able to represent dynamical systems of different complexities.
The derivations in this work are self-contained and the mathematically inclined neuroscientist can quickly get up to speed with the new multiplicative SCN framework, without the need for prior specific knowledge of SCNs. All the code is available and well commented in https://github.com/michnard/mult_synapses making this introduction even more accessible to its readers. This paper is relevant for those interested in neural representations of dynamical systems and the possible roles for multiplicative synapses and dendritic non-linearities. Those interested in neuromorphic computations will find here an efficient and direct way of representing non-linear dynamical systems (at least those well approximated by low-order polynomials). Finally, those interested in neural temporal pattern generators might find it surprising that only 10 integrate and fire neurons can already very reasonably approximate a chaotic Lorenz system.
 Nardin, M., Phillips, J. W., Podlaski, W. F., and Keemink, S. W. (2021) Nonlinear computations in spiking neural networks through multiplicative synapses. arXiv, ver. 4 peer-reviewed and recommended by Peer Community in Neuroscience. https://arxiv.org/abs/2009.03857v4
 Boerlin, M., Machens, C. K., and Denève, S. (2013). Predictive coding of dynamical variables in
balanced spiking networks. PLoS Comput Biol, 9(11):e1003258. https://doi.org/10.1371/journal.pcbi.1003258
Marco Leite (2021) Approximate any nonlinear system with spiking artificial neural networks: no training required. Peer Community in Neuroscience, 100003. 10.24072/pci.cneuro.100003
Evaluation round #2
DOI or URL of the preprint: https://arxiv.org/abs/2009.03857v3
Version of the preprint: v3
Author's Reply, 22 Nov 2021
Decision by Marco Leite, 05 Nov 2021
Thank you for your patience regarding the recommendation process, in normal circumstances it should not take this much time, and I apologize for that.
I am raising only a couple of minor points that need your attention:
- In section 5, in the new added text you write: "we have considered that each neuron codes for each dimension", while my understanding is that it should read "we have considered that each neuron codes for all dimensions", is this correct?
- The GitHub repository is missing the code for figure 4 and for the supplementary figures as well, it would be important to have them for completeness and reproducibility.
Once these are addressed I will be very happy to publish the my already prepared recommendation.
Evaluation round #1
DOI or URL of the preprint: https://arxiv.org/abs/2009.03857v2
Author's Reply, 04 Aug 2021
Decision by Marco Leite, 14 Jun 2021
Dear Michele Nardin,
Once again thank you for submiting your preprint for review in PCI C Neuro. We have gathererd the opinion of two reviewers and feel that the preprint represents a substantial contribution for the field of spiking neural networks. However, before proceeding with a recomendation, the reviewers have several comments that require your attention. Please consider addressing these point by point and submiting a revised version of the manuscript.