A Sparse Coding Neural Network ASIC With On-Chip Learning for Feature Extraction and Encoding Phil Knag , Student Member , IEEE, Jung Kuk Kim , Student Member , IEEE, Thomas Chen , Student Member , IEEE

Abstract

Hardware-based computer vision accelerators will be an essential part of future mobile devices to meet the low power and real-time processing requirement. To realize a high energy ef- ficiency and high throughput, the accelerator architecture can be massively parallelized and tailored to vision processing, which is an advantage over software-based s olutions and general-purpose hardware. In this work, we present an ASIC that is designed to learn and extract features from images and videos. The ASIC con- tains 256 leaky integrate-and-fire neurons connected in a scalable two-layer network of 8 8 grids linked in a 4-stage ring. Sparse neuron activation and the relatively small grid keep the spike colli- sion probability low to save access arbitration. The weight memory is divided into core memory and auxiliary memory, such that the auxiliary memory is only powered on for learning to save inference power. High-throughput inference is accomplished by the parallel operation of neurons. Efficient lea rning is implemented by passing parameter update messages, whic h is further simplified by an ap- proximation technique. A 3.06 mm 65 nm CMOS ASIC test chip is designed to achieve a maximum inference throughput of 1.24 Gpixel/s at 1.0 V and 310 MHz, and on-chip learning can be com- pleted in seconds. To improve the power consumption and energ y efficiency, core memory supply voltage can be reduced to 440 mV to take advantage of the error resi lience of the algorithm, reducing the inference p