A Quad-Chiplet AI SoC with Full-Chip Scalable Mesh Over 16Gb/s UCIe-Advanced Die-to-Die Interface for Large-Scale AI Inferencing

📋 论文概要

本文提出了一款基于4nm工艺的四芯粒AI SoC，通过先进的UCIe-Advanced芯片间接口实现全芯片可扩展网格，解决了大规模AI推理中芯片间带宽和能效瓶颈。在LLaMA模型上实现了56.8 TPS的吞吐量。

💡 主要创新点

• 采用四芯粒架构和全芯片可扩展网格，通过16Gb/s UCIe-Advanced接口实现高效芯片间通信。
• 在4nm工艺上集成LLM加速器，针对大规模AI推理优化能效和吞吐量。

🏷 关键词

📄 原文摘要

Sungpill Choi, Donghan Kim, Hyunje Jo, Hyunho Kim, Hyungseok Heo, Hyunsung Kim, Seung-Goo Kim, Myunghoon Choi, Sangeun Je, Junhee Ham, Juyeong Yoon, Yashael Faith Arthanto, Sung-il Bae, Sanggyu Park, Joungwoo Lee, Heeyoung Chae, Kiljun Ryu, Yongjik Kim, Jin-O Seo, Nara Cho, Taeyoung Jeon, Gukchae Ahn, Myoungoh Ki, Junkyeong Choi, Seungcheol Baek, Daehoon Kim, JuHwan Kim, Sungmoon Kang, Sieon Son, Minseo Kim, Yeounghwan Choe, Younggeun Lee, Sunghyun Park, Jinwook Oh Rebellions, Seongnam, Korea Abstract A 4nm-based quad-chiplet with an advanced packaged LLM accelerator achieving 56.8TPS on LLaMA v3.3 70B with single-batch 2k/2k input/output sequences. The architecture combines chiplet-based design, low-latency die-to-die interfaces, unified mixed-precision compute, holistic synchronization, and HBM3E with advanced power schemes to sustain bandwidth, capacity and thermal stability. The SoC integrates four NPU chiplets, four HBM3E

👥 作者与机构

Chang-Hyo Yu, Jaewan Bae, Jinseok Kim, Hongyun Kim, Wongyu Shin, Jae-Sung Yoon, Young-Jae Jin, Jungju Oh, Jaebong Lee, Eunseo Kim, Miock Chi,