A 65-nm Energy-Efficient Interframe Data Reuse Neural Network Accelerator for Video Applications Yixiong Y ang

Abstract

An energy-efficient convolutional neural network (CNN) accelerator is proposed for the video application. Pre- vious works exploited the sparsity of differential (Diff) frame activation, but the improvement is limited as many Diff-frame data is small but non-zero. Processing of irregular sparse data also leads to low hardware utilization. To solve these problems, two key innovations are proposed in this article. First, we implement a hybrid-precision inter-frame-reuse archi- tecture which takes advantage of both low bit-width and high sparsity of Diff-frame data. Th is technology can accelerate 3.2× inference speed with no accuracy loss. Second, we design a conv-pattern-aware processing array that achieves the 2.48×–14.2× PE utilization rate to process sparse data for differ- ent convolution kernels. The accelerator chip was implemented in 65-nm CMOS technology. To the best of our knowledge, it is the first silicon-proven CNN accelerator that supports inter-frame data reuse. Attributed to the inter-frame similarity, this video CNN accelerator reaches the minimum energy consumption of 24.7 µJ/frame in the MobileNet-slim model, which is 76.3% less than the baseline.