⚡ 本页包含 AI 生成的分析内容，仅供参考
提出了一种基于稀疏MoE的推测解码语言处理单元SMoLPU，通过Token自适应专家细化技术消除了冗余专家激活，并采用自适应卸载NPU-CIM核心来应对动态INT-FP比例，实现了低能耗和高硬件利用率。该处理器在每参数能耗上比现有最优技术降低了43.5%，达到了122.1μJ/Token的能耗效率。
decoding LLM processor with an NPU-CIM core. It has 3 features: 1) Token-adaptive expert refinement removes redundant expert activations and schedules expert load order, achieving 2.3×/4.2× energy efficiency improvement in prefill/decode; 2) An adaptive-offload NPU-CIM core and a top scheduling unit maintain HW utilization under dynamic INT–FP ratios, achieving 3.3× lower latency; 3) A reconfigurable DRAM-based LUT-CIM reduces adder tree power/area, while supporting dynamic input size. The LLM processor achieves 43.5% lower energy per parameter than prior SOTA. Reasoning strategies such as chain-of-thought (CoT) [1] improve the accuracy of large language models (LLMs) on complex tasks, but lengthen decoding sequences under limited parallelism. The longer sequences repeatedly fetch full model weights from external
Sangwoo Ha1, Jingu Lee1, Youngjin Moon1, Sunjoo Whang1, Wooyoung Jo1, Gwangtae Park1, Sangjin Kim1, Soyeon Um2, Junha Ryu1, Yurim Jo1, Hoi-Jun Yoo1
KAIST, Daejeon, Korea, 2Massachusetts Institute of Technology, Cambridge, MA 1 Abstract SMoLPU is an energy-efficient MoE-based speculative