revolutionized a wide range of AI applications, which motivates a surge in research to develop energy-efficient hardware accelerators. Most prior efforts have concentrated on enhancing on-chip computational energy efficiency through several strategies such as encoder-only models [1-7], quantization/sparsity [8-18], and layer pruning [19]. However, recent works [20,21] show that external memory access (EMA) dominates total energy consumption. Our analysis based on [22,23] also indicates that EMA accounts for up to 81% of the total energy usage (Fig. 23.1.1). Additionally, we recognize that the prior works exhibit low hardware utilization, as low as 9% in [4], which negatively impacts latency performance. In light of this, we present a novel transformer accelerator named T-REX to address the
Seunghyun Moon1, Mao Li1, Gregory K. Chen2, Phil C. Knag2,
Ram Kumar Krishnamurthy2, Mingoo Seok1 Columbia University, New York, NY Intel, Hillsboro, OR 1 2 Transformer, a recent mainstream model in deep learning, has