SambaNova SN40L: A 5nm 2.5D Dataflow Accelerator with Three Memory Tiers for Trillion Parameter AI

📋 论文概要

本文介绍了SambaNova SN40L，一款基于5nm工艺的2.5D数据流加速器，专为支持万亿参数AI模型设计。它通过三级存储系统（片上SRAM、封装HBM、片外DRAM）解决大模型的内存瓶颈，并采用CoWoS-S中介层集成两个加速器die与HBM。

💡 主要创新点

• 三级存储系统：结合片上SRAM、封装内HBM和片外DRAM，实现大模型高效数据访问。
• 2.5D CoWoS-S封装：集成两个相同加速器die和HBM内存模块，提升带宽和能效。
• 可重构数据流架构：在RDU基础上优化，支持灵活的数据流计算以适应不同AI工作负载。

🏷 关键词

📄 原文摘要

Mahmood Khayatzadeh, Kyunglok Kim, Uma Durairajan, Jeongha Park, Satyajit Sarkar, Jinuk Luke Shin SambaNova Systems, Palo Alto, CA The SN40L is the latest-generation Reconfigurable Dataflow Unit (RDU) from SambaNova Systems built for modern AI training and inference applications [1,2]. The SN40L combines on-chip dataflow flexibility with a three-tier memory system: on-chip SRAM, on-package High Bandwidth Memory (HBM), and off-package DRAM (Fig. 16.4.7). The SN40L employs 3× CoWoS-S interposer technology featuring two identical accelerator dies and HBM memory modules. To enable both high-bandwidth and high-capacity memory accesses from a single chip, along with peer-to-peer fabric and die-to-die connectivities, a dual-die approach is adopted, where the size of each die is 600mm2 (Fig. 16.4.1). Furthermore, a sub-reticle size dual-die design optimizes performance, thermal efficiency, and cost. The

👥 作者与机构

Raghu Prabhakar, Junwei Zhou, Darshan Gandhi, Youngmoon Choi,