Hardware Acceleration of LLMs: A comprehensive survey and comparison
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) have emerged as powerful tools for natural language processing tasks, revolutionizing the field with their ability to understand and generate human-like text. In this paper, we present a comprehensive survey of the several research efforts that have been presented for the acceleration of transformer networks for Large Language Models using hardware accelerators. The survey presents the frameworks that have been proposed and then performs a qualitative and quantitative comparison regarding the technology, the processing platform (FPGA, ASIC, In-Memory, GPU), the speedup, the energy efficiency, the performance (GOPs), and the energy efficiency (GOPs/W) of each framework. The main challenge in comparison is that every proposed scheme is implemented on a different process technology making hard a fair comparison. The main contribution of this paper is that we extrapolate the results of the performance and the energy efficiency on the same technology to make a fair comparison; one theoretical and one more practical. We implement part of the LLMs on several FPGA chips to extrapolate the results to the same process technology and then we make a fair comparison of the performance.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 24 of its 50 references that are in the map. First posted on arXiv 5 Sep 2024.
Builds on
24 of 50 references.
References · 24
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
- 2020A3: Accelerating Attention Mechanisms in Neural Networks with Approximationcited by 97
- 2020TurboTransformers: An Efficient GPU Serving System For Transformer Modelscited by 71
- 2023Inference with Reference: Lossless Acceleration of Large Language Modelscited by 63
- 2020FTRANS: Energy-Efficient Acceleration of Transformers using FPGAcited by 52
- 2024ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Cachingcited by 48
- 2022DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generationcited by 46
- 2020Hardware Accelerator for Multi-Head Attention and Position-Wise Feed-Forward in the Transformercited by 42
- 2023Simplifying Transformer Blockscited by 37
- 2021Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attentioncited by 26
- 2023X-Former: In-Memory Acceleration of Transformerscited by 25
12 more
- 2023AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformerscited by 24
- 2021LightSeq2: Accelerated Training for Transformer-based Models on GPUscited by 19
- 2021NPE: An FPGA-based Overlay Processor for Natural Language Processingcited by 15
- 2022SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequencescited by 10
- 2024SSR: Spatial Sequential Hybrid Architecture for Latency Throughput Tradeoff in Transformer Accelerationcited by 9
- 2023Transformer-Based Autonomous Driving Models and Deployment-Oriented Compression: A Surveycited by 8
- 2023A Comprehensive Performance Study of Large Language Models on Novel AI Acceleratorscited by 7
- 2024BETA: Binarized Energy-Efficient Transformer Accelerator at the Edgecited by 6
- 2024TransAxx: Efficient Transformers with Approximate Computingcited by 6
- 2024ME-ViT: A Single-Load Memory-Efficient FPGA Accelerator for Vision Transformerscited by 6
- 2023An Efficient FPGA-Based Accelerator for Swin Transformercited by 5
- 2024A Cost-Efficient FPGA Implementation of Tiny Transformer Model using Neural ODEcited by 4
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 2.
3 papers.