A 2.1TFLOPS/W Mobile Deep RL Accelerator with Transposable PE Array and Experience Compression

📋 论文概要

该论文提出了一种用于移动深度强化学习的加速器芯片，通过可转置处理单元阵列和体验压缩技术，实现了高能效的实时推理和训练，解决了强化学习在自主系统中需要本地实时操作的问题。

💡 主要创新点

• 提出可转置PE阵列，支持矩阵转置操作，提升强化学习训练中的计算效率。
• 采用体验压缩技术，减少经验回放缓冲区的存储和带宽需求。
• 整体架构面向移动端实现2.1TFLOPS/W的高能效比。

🏷 关键词

📄 原文摘要

but also for action control, so that an autonomous system, such as the robot, can perform human-like behaviors and operations. Unlike recognition tasks, real-time operation is important in action control, and it is too slow to use remote learning on a server communicating through a network. New learning techniques, such as reinforcement learning (RL), are needed to determine and select the correct robot behavior locally. Fig. 7.4.1(a) shows an example of a robot agent that uses a pretrained DNN without RL, and Fig. 7.4.1(b) depicts an autonomous robot agent that learns continuously in the environment using RL. The agent without RL falls down if the land slope changes, but the RL-based agent iteratively collects walking experiences and learns to walk even though the land slope changes.

👥 作者与机构

Changhyeon Kim, Sanghoon Kang, Dongjoo Shin, Sungpill Choi,

Youngwoo Kim, Hoi-Jun Yoo KAIST, Daejeon, Korea Recently, deep neural networks (DNNs) are actively used for object recognition,