A 4096-Neuron 1M-Synaps e 3.8-pJ/SOP Spiking Neural Network With On-Chip STDP Learning and Sparse Weights in 10-nm FinFET CMOS

Abstract

A reconfigurable 4096-neuron, 1M-synapse chip in 10-nm FinFET CMOS is developed to accelerate inference and learning for many classes of spiking neural networks (SNNs). The SNN features digital circuits for leaky integrate and fire neuron models, on-chip spike-timing-dependent plasticity (STDP) learning, and high-fan-out multicast spike communication. Struc- tured fine-grained weight sparsity reduces synapse memory by up to 16 × with less than 2% overhead for storing connections. Approximate computing co-optimizes the dropping flow control and benefits from algorithmic noise to process spatiotemporal spike patterns with up to 9.4 × lower energy. The SNN achieves a peak throughput of 25.2 GSOP/s at 0.9 V , peak energy efficiency of 3.8 pJ/SOP at 525 mV , and 2.3- µW/neuron oper- ation at 450 mV . On-chip unsupervised STDP trains a spiking restricted Boltzmann machine to de-noise Modified National Institute of Standards and Technology (MNIST) digits and to reconstruct natural scene images with RMSE of 0.036. Near- threshold operation, in conjunction with temporal and spatial sparsity, reduces energy by 17.4 × to 1.0- µJ/classification in a 236 × 20 feed-forward network that is trained to classify MNIST digits using supervised STDP. A binary-activation multilayer perceptron with 50% sparse weights is trained offline with error backpropagation to classify MNIST digits with 97.9% accuracy at 1.7- µJ/classification.