MetaVRain: A 133mW Real-Time Hyper-Realistic 3D-NeRF Processor with 1D-2D Hybrid-Neural Engines for Metaverse on Mobile Devices

📋 论文概要

本文提出MetaVRain，一种用于移动端元宇宙的实时超逼真3D NeRF处理器，功耗仅133mW。通过设计1D-2D混合神经网络引擎，实现了高效低功耗的NeRF推理，解决了NeRF在移动设备上实时运行的难题。

💡 主要创新点

• 提出1D-2D混合神经网络引擎，结合1D和2D计算优势，降低计算复杂度
• 实现133mW超低功耗下的实时3D NeRF渲染，适用于移动元宇宙场景
• 支持低带宽参数传输（<1MB），便于多用户共享3D模型

🏷 关键词

📄 原文摘要

A neural radiance field (NeRF) [1] uses a deep neural network (DNN) to create 3D models by training the DNN to memorize 3D scene geometry from a few photos. Prior work uses conventional computer graphic algorithms, such as ray-tracing or SLAM, for the same purpose. With NeRF, the generated model can display hyper-realistic 3D content on the metaverse, with quality better than or the same as 3D images rendered by complicated ray-tracing. The 3D model can also be shared with other metaverse users via lowbandwidth communication because the transaction requires <1MB of parameters. NeRF is promising for 3D reconstruction, but also for a wide range of applications from depth estimation to 3D style transfer [2], however, its heavy computational demands stand in the way of its applicability for mobile and wearable applications. Figure 2.7.1 shows an overview of NeRF and the proposed Bundle-Frame-Familiarity

👥 作者与机构

Donghyeon Han, Junha Ryu, Sangyeob Kim, Sangjin Kim, Hoi-Jun Yoo

Korea Advanced Institute of Science and Technology, Daejeon, Korea