VARSA: A Visual Autoregressive Generation Accelerator Using Performance-Scalable Multi-Precision PE-LUT and Grid-Similarity Attention Compression

📋 论文概要

本文提出VARSA，一种22nm工艺的视觉自回归生成加速器，用于高效文本到图像生成。通过性能可扩展的混合PE-LUT核心、多精度并行处理以及注意力图压缩技术，实现了503mJ/inference的能效，比现有扩散模型加速器提升2.7-8.9倍。

💡 主要创新点

• 提出性能可扩展的混合PE-LUT核心，兼顾计算灵活性与效率。
• 实现多精度并行处理与运行时精度管理，适应不同生成阶段需求。
• 利用网格间相似性进行注意力图压缩，减少计算和存储开销。

🏷 关键词

📄 原文摘要

Abstract This paper presents VARSA, a 22nm visual autoregressive accelerator for efficient text-toimage generation, featuring: 1) a performance-scalable hybrid PE-LUT core; 2) multi-precision parallel processing with runtime precision management; 3) attention map compression leveraging inter-grid similarity. The innovations enable VARSA to achieve 503mJ/inference for 512×512 image generation, which is 2.7-to-8.9× better than prior diffusion-based SOTA accelerators. AI-generated content (AIGC) has become an essential technique to support various content generation, e.g., images, videos. In recent years, the model architecture of visual AIGC has rapidly evolved from the Diffusion Transformer (DiT) [1-3], to the autoregressive (AR) Transformer [4-6] to the visual autoregressive model (VAR) [7-10]. As shown in Fig. 31.4.1, DiT or AR models require either extensive denoising timesteps or inefficient serial rasterscan, leading to high inference latency. For example, generating a 512×512 imag

👥 作者与机构

Jiaqi Zhou, Hongou Li, Keyao Jiang, Yiyang Sun, Tianyu Jia

Peking University, Beijing, China