# Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml

arXiv: https://arxiv.org/abs/2409.05207 · HTML: https://mlsanity.com/paper/2409.05207/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 8 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors](https://mlsanity.com/paper/2006.10159/) (2020)
- [FTRANS: Energy-Efficient Acceleration of Transformers using FPGA](https://mlsanity.com/paper/2007.08563/) (2020)
- [DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation](https://mlsanity.com/paper/2209.10797/) (2022)
- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Graph Neural Networks for Charged Particle Tracking on FPGAs](https://mlsanity.com/paper/2112.02048/) (2021)
- [Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml](https://mlsanity.com/paper/2207.00559/) (2022)
- [Accelerating Recurrent Neural Networks for Gravitational Wave Experiments](https://mlsanity.com/paper/2106.14089/) (2021)

## Built on by (most cited first, 5 of 5)

- [hls4ml: A Flexible, Open-Source Platform for Deep Learning Acceleration on Reconfigurable Hardware](https://mlsanity.com/paper/2512.01463/) (2025)
- [JetFormer: A Scalable and Efficient Transformer for Jet Tagging from Offline Analysis to FPGA Triggers](https://mlsanity.com/paper/2601.17215/) (2026)
- [SparsePixels: Efficient Convolution for Sparse Data on FPGAs](https://mlsanity.com/paper/2512.06208/) (2025)
- [Taming the Exponential: A Fast Softmax Surrogate for Integer-Native Edge Inference](https://mlsanity.com/paper/2604.02292/) (2026)
- [Recent Developments in Transformer Inference Deployment on FPGA Platforms: A Survey](https://mlsanity.com/paper/2609.01212/) (2026)

## Abstract (arXiv)

This study presents an efficient implementation of transformer architectures in Field-Programmable Gate Arrays(FPGAs) using hls4ml. We demonstrate the strategy for implementing the multi-head attention, softmax, and normalization layer and evaluate three distinct models. Their deployment on VU13P FPGA chip achieved latency less than 2us, demonstrating the potential for real-time applications. HLS4ML compatibility with any TensorFlow-built transformer model further enhances the scalability and applicability of this work. Index Terms: FPGAs, machine learning, transformers, high energy physics, LIGO