A 51.6TFLOPs/W Full-Datapath CIM Macro Approaching Sparsity Bound and <2-30 Loss for Compound AI

📋 论文概要

本文提出了一种全数据通路（Full-Datapath）的SRAM浮点计算存内（CIM）宏，针对复合AI（Compound AI）系统实现高效边缘部署。该宏能效达51.6 TFLOPs/W，接近稀疏性理论界限，且精度损失小于2-30，解决了传统大模型在边缘设备上部署的成本和尺寸问题。

💡 主要创新点

• 提出全数据通路CIM架构，支持浮点运算，实现高能效边缘AI加速。
• 通过接近稀疏性界限的设计，在保持高精度的同时大幅提升能效至51.6 TFLOPs/W。
• 针对复合AI系统优化，结合多个小模型实现与大型模型相当甚至更优的精度。

🏷 关键词

📄 原文摘要

with exceptional performance, but their prohibitive size and cost limits deployment on edge devices. The compound-AI combines several specialized small models to achieve matched or even superior accuracy on target downstream tasks [1,2]: e.g., RevCol [3] fuses multiple convolutional models and surpasses the 7.2B-parameter monolithic LLM model [4] on the ImageNet classification task by 1.64%. Therefore, the shift to compound systems opens opportunities for edge deployment in addition to model-size scaling. SRAM-based floating point (FP) CIM promises accelerated edge-AI models [5-14]. Figure 14.4.1 shows a conventional FP CIM macro, which faces three challenges for compound-AI acceleration: (1) Previous FP CIMs rely on a Gaussian data distribution to reduce alignment loss, where data is distributed around its mean [11].

👥 作者与机构

Zhiheng Yue*, Xujiang Xiang*, Yang Wang, Ruiqi Guo, Huiming Han,

Shaojun Wei, Yang Hu, Shouyi Yin Tsinghua University, Beijing, China *Equally Credited Authors (ECAs) Large-language models (LLM) are widely used