A 40nm 4.81TFLOPS/W 8b Floating-Point Training Processor for Non-Sparse Neural Networks Using Shared Exponent Bias and 24-Way Fused Multiply-Add Tree

📋 论文概要

针对非稀疏神经网络训练需求，提出一款采用40nm工艺的8位浮点训练处理器，通过共享指数偏置技术实现4.81TFLOPS/W高能效，解决了现代非稀疏激活函数下传统稀疏加速方法失效的问题。

💡 主要创新点

• 提出专为非稀疏神经网络设计的8位浮点训练处理器架构
• 采用共享指数偏置技术减少数据表示冗余，提升能效
• 在40nm工艺下实现4.81TFLOPS/W的领先能效

🏷 关键词

📄 原文摘要

*Equally Credited Authors (ECAs) Recent works on mobile deep-learning processors have presented designs that exploit sparsity [2, 3], which is commonly found in various neural networks. However, due to the shift in the machine learning community towards using non-sparse activation functions such as Leaky ReLU or Swish for better training convergence, state-of-theart models no longer exhibit the sparsity found in conventional ReLU-based models (Fig. 9.3.1, top). Moreover, contrary to error-tolerant image classification tasks, more difficult tasks such as image super-resolution require higher precision than plain 8b integers not just for training, but for inference without large accuracy degradation (Fig. 9.3.1, bottom). These changes offer new challenges faced by mobile deep-learning processors: they must process non-sparse networks efficiently and maintain higher precision for more challenging tasks.

👥 作者与机构

Jeongwoo Park*, Sunwoo Lee*, Dongsuk Jeon

Seoul National University, Seoul, Korea