A 9.02mW CNN-Stereo-Based Real-Time 3D Hand-Gesture Recognition Processor for Smart Mobile Devices

📋 论文概要

本文提出一种基于CNN和立体匹配的实时3D手势识别处理器，功耗仅9.02mW，适用于头戴显示器等智能移动设备。该处理器解决了传统ToF深度传感器功耗高（>2W）导致续航不足的问题，同时克服了纯立体匹配方案交互性差的缺点。

💡 主要创新点

• 提出将CNN与立体匹配相结合的新型算法，实现低功耗且高精度的深度估计。
• 设计专用硬件处理器架构，在9.02mW超低功耗下完成实时3D手势识别。
• 针对智能移动设备（如AR/VR头显）优化，支持实时交互且功耗比传统ToF方案降低两个数量级。

🏷 关键词

📄 原文摘要

Recently, 3D hand-gesture recognition (HGR) has become an important feature in smart mobile devices, such as head-mounted displays (HMDs) or smartphones for AR/VR applications. A 3D HGR system in Fig. 13.4.1 enables users to interact with virtual 3D objects using depth sensing and hand tracking. However, a previous 3D HGR system, such as Hololens [1], utilized a power consuming timeof-flight (ToF) depth sensor (>2W) limiting 3D HGR operation to less than 3 hours. Even though stereo matching was used instead of ToF for depth sensing with low power consumption [2], it could not provide interaction with virtual 3D objects because depth information was used only for hand segmentation. The HGR-based UI system in smart mobile devices, such as HMDs, must be low power consumption (<10mW), while maintaining real-time operation (<33.3ms). A convolutional neural network (CNN) can be adopted to enhance the accuracy of

👥 作者与机构

Sungpill Choi, Jinsu Lee, Kyuho Lee, Hoi-Jun Yoo

KAIST, Daejeon, Korea