184QPS/W 64Mb/mm2 3D Logic-to-DRAM Hybrid Bonding with Process-Near-Memory Engine for Recommendation System

📋 论文概要

提出了一种采用3D逻辑与DRAM混合键合技术的近存储计算引擎，专为推荐系统设计，实现了184QPS/W的能效和64Mb/mm2的存储密度，解决了AI系统中内存带宽和容量瓶颈问题。

💡 主要创新点

• 首次在推荐系统应用中采用3D Logic-to-DRAM混合键合技术，实现了高密度高带宽的存储集成。
• 设计了近存储处理引擎，通过近存储计算大幅降低数据搬运能耗，达到184QPS/W的极致能效。
• 展示了64Mb/mm2的存储密度，优于传统方案，为大规模推荐模型提供片上存储支持。

🏷 关键词

📄 原文摘要

Tianchan Guan3, Fei Sun1, Fei Xue1, Lide Duan1, Yuanwei Fang1, Hongzhong Zheng1, Xiping Jiang4, Song Wang4, Fengguo Zuo4, Yubing Wang4, Bing Yu4, Qiwei Ren4, Yuan Xie1 Alibaba DAMO Academy, Sunnyvale, CA; 2Alibaba DAMO Academy, Beijing, China Alibaba DAMO Academy, Shanghai, China; 4UniIC, Xian, China 1 3 The era of AI computing brings significant challenges to traditional computer systems. As shown in Fig. 29.1.1, while the AI model computation requirement increases 750× every two years, we only observe a very slow-paced improvement of memory system capability in terms of both capacity and bandwidth. There are many memory-bound applications, such as natural language processing, recommendation systems, graph analytics, graph neural networks, as well as multi-task online inference, that become dominating AI applications in modern cloud datacenters. Current primary memory technologies that power AI systems and applications include on-chip memory (SRAM),

👥 作者与机构

Dimin Niu1, Shuangchen Li1, Yuhao Wang1, Wei Han1, Zhe Zhang2, Yijin Guan2,