Hybrid SRAM/ROM Compute-in-Memory Architecture for High Task-Level Energy Efficiency in Transformer Models With 8928-kb/mm 2 Density in 28nm CMOS

Abstract

This article reports the first ultra-high-density hybrid static random access memory (SRAM)/read-only mem- ory (ROM) compute-in-memory (CiM) architecture for multibit multiply-accumulate (MAC) operations of transformer models. It features several techniques that enhance the on-chip weight density, energy efficiency, and flexibility, namely: 1) the hybrid CiM structure of 6T SRAM and 1T multi-level cell (MLC) ROM for ultra-high memory density to improve the limited on-chip memory capacity and alleviate the frequent weight reload from dynamic random access memory (DRAM); 2) 2- /5-b adaptive-resolution analog-to-digital converters (ADCs) for energy reduction and low accuracy loss; and 3) post-fabrication 1-of-4 capacitor selection (PFCS) to reduce the capacitance of computing capacitors, which was originally limited by the capacitor variation. This leads to lower energy and higher accuracy. A record-high 22-Mb ROM CiM and 896-kb SRAM CiM macro has demonstrated the 8 × 8 b MAC operations of one transformer layer, featuring ultra-high weight density of 8928 kb/mm 2 and significantly reduced DRAM access activities toward high inference energy efficiency.