An Efficient Deep-Learning-Based Super-Resolution Accelerating SoC With Heterogeneous Accelerating and

Abstract

This article presents an energy-efficient accelerating system-on-chip (SoC) for super-resolution (SR) image reconstruc- tion on a mobile platform. With the rise of contactless commu- nication and streaming services, the need for SR is growing. As one of the most basic low-level image processing algorithms, SR can reconstruct high-quality images from low-quality images which are noisy, compressed, or with damaged pixels. However, a massive amount of computation and considerable precision of pixel data pose challenges for acceleration in a resource and bandwidth constrained platform. SR has high energy consump- tion and long latency. While previous neural processing units (NPUs) reduced the precision to increase the efficiency and accelerate convolutional neural network (CNN) computation, few of them concentrated on both the output image quality and the performance of the entire system. The proposed SR SoC restores the high-quality image using a precision-optimized SR algorithm on an energy-efficient accelerating architecture and cache subsystem. It contributes three algorithm-hardware co- optimized features: 1) heterogeneous accelerating architecture (HAA) with only 8-bit floating-point (FP)-and-fixed-point (FXP) hybrid-precision for SR task; 2) tile-based hierarchical cache (THC) subsystem for the low energy and small footprint cost layer fusion; and 3) heterogeneous L1 data lifetime-aware optimized cache (DLOC) for the energy-efficient on-chip memory access. The prototype of SR So