Slim-Llama: A 4.69mW Large-Language-Model Processor with Binary/Ternary Weights for Billion-Parameter Llama Model

📋 论文概要

针对大型语言模型外部存储器访问能耗过高的问题，提出了一款名为Slim-Llama的处理器，支持二元/三元权重，可处理十亿参数级别的Llama模型，功耗仅4.69mW。

💡 主要创新点

• 采用二元/三元权重大幅降低模型参数存储和访问能耗，从而减少外部存储器访问次数。
• 面向十亿参数Llama模型进行架构优化，在极低功耗下实现大模型推理。

🏷 关键词

📄 原文摘要

Recently, multiple ASICs [1-6] have been proposed to accelerate large language models (LLMs). However, the enormous number of LLM parameters leads to significant energy consumption due to external memory access (EMA). When normalizing the system energy required to process 1024 input tokens by the number of parameters, previous ASICs [1-5] required 79-222pJ/param for small models with 336-682M parameters, as shown in Fig. 23.9.1. Even an ASIC [6] designed to reduce EMA still consumes 26pJ/param for a 708M model. Due to large EMA, prior works [1-6] were limited to processing small models like GPT-2 [7] and cannot handle highly accurate AI models with billions of parameters, such as Llama [8]. To overcome this, we adopt binary or ternary quantization and new hardware optimizations, enabling the proposed ASIC to consume 9pJ/param and less than 5mW power for a billion-parameter Llama. Recently, binary/ternary weight quantization [9,10] for LLMs has been proposed, replacing

👥 作者与机构

Sangyeob Kim, Jungwan Lee, Hoi-Jun Yoo

KAIST, Daejeon, Korea