A 1.2GHz 12.77GB/s/mm2 3D Two-DRAM-One-Logic Process-Near-Memory Chip for Edge LLM Applications

📋 论文概要

本文提出了一种面向边缘大语言模型应用的高带宽密度、高存储密度、低能耗的3D近存计算芯片，采用双DRAM单逻辑架构，通过3D集成实现近DRAM计算，显著降低内存访问延迟和通用矩阵乘法执行时间。

💡 主要创新点

• 提出双DRAM单逻辑的3D近存计算架构，实现高密度3D集成路径。
• 通过近DRAM计算技术，将内存访问延迟降低93%，GEMM执行时间降低98%。

🏷 关键词

📄 原文摘要

Sciences, Beijing, China, 3Zhangjiang Laboratory, Shanghai, China Xi’an UniIC Semiconductors, Xi’an, China 1 4 Abstract A high-bandwidth-density (12.77GB/s/mm2) high-memory-density (99.4Mb/mm2) lowenergy-consumption (0.67pJ/b) 3D PNM design that operates at 1.2GHz is presented. The design adopts a two-DRAM-one-logic architecture that enables near-DRAM computing through a high-density 3D integration path, reducing memory-access latency by up to 93% and GEMM execution time by up to 98%, demonstrating strong potential for edge-LLM workloads. Upon the rapid development of generative AI, there is an urgent need to embed LLM inference support into client devices (PCs, mobiles) to improve response time, data privacy, and customization. However, as shown in Fig. 30.7.1, challenges remain for edge-LLM

👥 作者与机构

Yue Cao1,2,3, Jinghao Jiang1, Haijun Jiang3, Qian Zhang3, Xuanzhi Liu2, Jinhui Cheng2, Zhongze Han2, Xiping Jiang4, Fengguo Zuo4, Song Wang4, Fujun Bai4,

Yixin Guo4, Chunmeng Dou2, Jianguo Yang1,2,3, Hangbing Lv2, Qi Liu1, Ming Liu1,2 Fudan University, Shanghai, China, 2Institute of Microelectronics of the Chinese Academy of