A Sub-1µJ/class Headset-Integrated Mind Imagery and Control SoC for VR/MR Applications with Teacher-Student CNN and General-Purpose Instruction Set Architecture

📋 论文概要

该论文提出了一款头戴式集成的心像与控制SoC，用于VR/MR应用，采用教师-学生CNN架构实现极低功耗的脑信号解码，每次分类能耗低于1微焦耳。解决了现有VR头显依赖传统摇杆或摄像头手势控制的局限，实现了基于思维想象的直接交互。

💡 主要创新点

• 集成教师-学生CNN架构，实现高能效的脑信号分类，降低计算功耗。
• 将心像（mind imagery）解码功能集成到头戴设备SoC中，实现低延迟、低功耗的脑机接口。
• 每次分类能量低于1µJ，适合消费级VR/MR设备的电池供电场景。

🏷 关键词

📄 原文摘要

*Equally Credited Authors (ECAs) Virtual Reality (VR) and Mixed Reality (MR) systems, e.g., Meta Quest and Apple Vision Pro, have recently gained significant interest in consumer electronics, creating a new wave of developments in metaverse for gaming, social networking, workforce assistance, online shopping, etc. Strong technological innovations in AI computing and multimodular human activity tracking and control have produced immersive virtual realistic user experiences. However, most existing VR headsets only rely on traditional joysticks or camera-based user gestures for input control and human tracking, missing an important source of information, namely, brain activity. Hence there is a growing interest in incorporating brain-machine interfaces (BMIs) into VR/MR systems for consumer and clinical applications [1]. As illustrated in Fig. 33.2.1, an existing VR/MR system integrated with EEG channels typically consists of a VR headset, a 16/32-channel EEG cap, a neural

👥 作者与机构

Zhiwei Zhong*, Yijie Wei*, Lance Christopher Go, Jie Gu

Northwestern University, Evanston, IL