mlsanityData to 7 Oct 2026

Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml

arXiv 2409.05207 ↗First posted 8 Sep 2024cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

This study presents an efficient implementation of transformer architectures in Field-Programmable Gate Arrays(FPGAs) using hls4ml. We demonstrate the strategy for implementing the multi-head attention, softmax, and normalization layer and evaluate three distinct models. Their deployment on VU13P FPGA chip achieved latency less than 2us, demonstrating the potential for real-time applications. HLS4ML compatibility with any TensorFlow-built transformer model further enhances the scalability and applicability of this work. Index Terms: FPGAs, machine learning, transformers, high energy physics, LIGO

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 9 of its 22 references that are in the map. First posted on arXiv 8 Sep 2024.

Builds on

9 of 22 references.

References · 9

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Fast inference of deep neural networks in FPGAs for particle physicscited by 132
  3. 2020Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectorscited by 78
  4. 2020FTRANS: Energy-Efficient Acceleration of Transformers using FPGAcited by 52
  5. 2022DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generationcited by 46
  6. 2021Fast convolutional neural networks on FPGAs with hls4mlcited by 42
  7. 2021Graph Neural Networks for Charged Particle Tracking on FPGAscited by 17
  8. 2022Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4mlcited by 14
  9. 2021Accelerating Recurrent Neural Networks for Gravitational Wave Experimentscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardwarecited by 8
  2. 2026JetFormer: A Scalable and Efficient Transformer for Jet Tagging from Offline Analysis to FPGA Triggerscited by 3
  3. 2025SparsePixels: Efficient Convolution for Sparse Data on FPGAscited by 0
  4. 2026Taming the Exponential: A Fast Softmax Surrogate for Integer-Native Edge Inferencecited by 0
  5. 2026Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Surveycited by 0