⚡ 本页包含 AI 生成的分析内容，仅供参考
本文提出了一种面向大型语言模型（LLM）的高能效加速器，通过位级权重压缩和簇对齐的INT-FP混合精度矩阵乘法（GEMM）技术，解决了LLM推理中巨大的外部内存访问和自回归计算带来的效率瓶颈。该加速器实现了88.36 TOPS/W的能效，显著降低了能耗。
range of natural language processing (NLP) tasks, becoming an essential part of modern society [1-4]. This exceptional performance can be attributed to huge model size and autoregressive computation [5,6]. However, these attributes pose challenges for the efficient deployment of LLMs from 3 aspects, as shown in Fig. 23.8.1. First, an LLM has enormous external memory access (EMA) due to its autoregressive feature. During inference, it generates output tokens one by one until all the outputs are complete. In each iteration, the processor reads all model weights and the key-value (KV) cache for computation, while also storing a newly generated KV into the cache for further use. Hence, an LLM requires over 1000× EMA compared to a non-autoregressive Transformer [7,8]. Second, an LLM may use a
Yubin Qin, Yang Wang, Jiachen Wang, Zhiwei Lin, Yushu Zhao, Shaojun Wei,
Yang Hu, Shouyi Yin Tsinghua University, Beijing, China Large language models (LLMs) have shown remarkable performance across a wide