An On-Device Generative AI Focused Neural Processing Unit in 4nm Flagship Mobile SoC with Fan-Out Wafer-Level Package

📋 论文概要

本文介绍了一款面向设备端生成式AI的神经处理单元（NPU），集成在4nm旗舰移动SoC中，采用扇出晶圆级封装。该NPU针对从CNN到Transformer的演进趋势进行优化，支持大语言模型（如LLaMA）和大视觉模型，实现自然语言理解与文本生成等功能。

💡 主要创新点

• 专为生成式AI设计，针对Transformer架构进行硬件加速，以高效支持大语言模型和大视觉模型推理。
• 采用4nm先进工艺和扇出晶圆级封装，在旗舰移动SoC中实现高集成度与性能。

🏷 关键词

📄 原文摘要

Mookyung Kang, Heeseok Lee, Jinwon Kang, Taeho Jeon, Dongwoo Lee, Yesung Kang, Kyungmok Kum, Geunwon Lee, Hongki Lee, Minkyu Kim, Suknam Kwon, Sung-beom Park, Dongkeun Kim, Chulmin Jo, HyukJun Chung, Ilryoung Kim , Jongyoul Lee Samsung Electronics, Hwaseong, Korea A notable trend observed in on-device AI is natural progression from camera applicationcentric CNN-based neural networks to transformer-based generative AI (Gen AI) [1]. For instance, large language models (LLM) such as LLaMA [2] can support natural language understanding and human-like text generation while large visual models (LVM) such as Stable Diffusion [3] can generate images or 3D models based on user context. However, gen AI models exhibit different operational characteristics from traditional neural network (NN) models. LLMs require reading a several GB of weight data from DRAM every time a single token is generated during decoding, resulting in memory-intensive behavior. LVMs,

👥 作者与机构

Jun-Seok Park, Taehee Lee, Heonsoo Lee, Changsoo Park, Youngsang Cho,