A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding

📋 论文概要

本文提出一款55nm基于推测解码的LLM加速器，采用bumping-based face-to-face ReRAM-on-logic堆叠技术。通过本地旋转单元实现无异常值低比特量化，并设计堆叠感知的PNM架构与块状向量量化协同降低权重开销，结合自适应并行推测解码和乱序调度器提升资源利用率，最终实现14.08-135.69 token/s的吞吐性能。

💡 主要创新点

• 提出本地旋转单元（Local Rotation Unit），在低比特量化中消除异常值，保证精度。
• 设计堆叠感知的PNM架构与块状向量量化（Blockwise Vector Quantization）协同，减少权重外部存储器访问开销。
• 实现自适应并行推测解码（Adaptive Parallel Speculative Decoding）与乱序调度器，提高硬件资源与带宽利用率。

🏷 关键词

📄 原文摘要

System, Hong Kong, China, 3Hefei Reliance Memory, Hefei, China, Zhejiang University, Hangzhou, China 1 4 Abstract This work presents a 55nm speculative decoding-based LLM accelerator with bumpingbased face-to-face ReRAM-on-logic stacking technology. It features a local rotation unit for outlier-free low-bit quantization, a stacking-aware PNM architecture co-designed with blockwise vector quantization to reduce weight EMA overheads, and an adaptive parallel speculative decoding scheme with out-of-order scheduler for high resource and bandwidth utilization. Our chip achieves 14.08-to-135.69token/s and 4.46-to-7.17× speedup over vanilla speculative decoding. Large Language Models (LLMs) [1-2] have achieved exceptional performance in natural

👥 作者与机构

Pingcheng Dong1,2, Yonghao Tan1,2, Xuejiao Liu2, Peng Luo2, Yu Liu2, Di Pang2, Songchen Ma1,2, Xijie Huang1, Shih-Yang Liu1, Dong Zhang1,2, Zhichao Lu3,

Luhong Liang2, Chi-Ying Tsui1,2, Fengbin Tu1,2, Liang Zhao4, Kwang-Ting Cheng1,2 Hong Kong University of Science and Technology, Hong Kong, China, 2AI Chip Center for Emerging Smart