mlsanityData to 14 Oct 2026

BETA: Binarized Energy-Efficient Transformer Accelerator at the Edge

arXiv 2401.11851 ↗First posted 22 Jan 2024cs.AR cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Existing binary Transformers are promising in edge deployment due to their compact model size, low computational complexity, and considerable inference accuracy. However, deploying binary Transformers faces challenges on prior processors due to inefficient execution of quantized matrix multiplication (QMM) and the energy consumption overhead caused by multi-precision activations. To tackle the challenges above, we first develop a computation flow abstraction method for binary Transformers to improve QMM execution efficiency by optimizing the computation order. Furthermore, a binarized energy-efficient Transformer accelerator, namely BETA, is proposed to boost the efficient deployment at the edge. Notably, BETA features a configurable QMM engine, accommodating diverse activation precisions of binary Transformers and offering high-parallelism and high-speed for QMMs with impressive energy efficiency. Experimental results evaluated on ZCU102 FPGA show BETA achieves an average energy efficiency of 174 GOPS/W, which is 1.76~21.92x higher than prior FPGA-based accelerators, showing BETA's good potential for edge Transformer acceleration.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 25 of its 30 references that are in the map. First posted on arXiv 22 Jan 2024.

Builds on

25 of 30 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  6. 2019Q8BERT: Quantized 8Bit BERTcited by 300
  7. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 191
  8. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  9. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  10. 2020TernaryBERT: Distillation-aware Ultra-low Bit BERTcited by 148
  11. 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 106
  12. 2020A3: Accelerating Attention Mechanisms in Neural Networks with Approximationcited by 101
13 more
  1. 2022BiBERT: Accurate Fully Binarized BERTcited by 73
  2. 2020FTRANS: Energy-Efficient Acceleration of Transformers using FPGAcited by 54
  3. 2022BiT: Robustly Binarized Multi-distilled Transformercited by 52
  4. 2020Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformercited by 43
  5. 2022An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformerscited by 29
  6. 2022VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformercited by 27
  7. 2021Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processingcited by 22
  8. 2023SwiftTron: An Efficient Hardware Accelerator for Quantized Transformerscited by 18
  9. 2023BinaryViT: Pushing Binary Vision Transformers Towards Convolutional Modelscited by 17
  10. 2023ShiftAddViT: Mixture of Multiplication Primitives Towards Efficient Vision Transformercited by 15
  11. 2022BEBERT: Efficient and Robust Binary Ensemble BERTcited by 14
  12. 2023ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformerscited by 12
  13. 2023ViTA: A Vision Transformer Inference Accelerator for Edge Applicationscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2025COBRA: Algorithm-Architecture Co-optimized Binary Transformer Accelerator for Edge Inferencecited by 4
  2. 2024Hardware Acceleration of LLMs: A comprehensive survey and comparisoncited by 3
  3. 2025Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimizationcited by 3
  4. 2026A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inferencecited by 1
  5. 2025NeuronMLP: Efficient LLM Inference via Singular Value Decomposition Compression and Tiling on AWS Trainiumcited by 0
  6. 2026Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Co-Designing Binarized Transformer and Hardware Accelerator for Efficient End-to-End Edge Deployment
  2. 2025QUARK: Quantization-Enabled Circuit Sharing for Transformer Acceleration by Exploiting Common Patterns in Nonlinear Operations
  3. 2025SeVeDo: A Heterogeneous Transformer Accelerator for Low-Bit Inference via Hierarchical Group Quantization and SVD-Guided Mixed Precision
  4. 2025Atleus: Accelerating Transformers on the Edge Enabled by 3D Heterogeneous Manycore Architectures
  5. 2024TATAA: Programmable Mixed-Precision Transformer Acceleration with a Transformable Arithmetic Architecture
  6. 2026BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design
  7. 2026ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator
  8. 2025ESACT: An End-to-End Sparse Accelerator for Compute-Intensive Transformers via Local Similarity