mlsanityData to 7 Oct 2026

LoopLynx: A Scalable Dataflow Architecture for Efficient LLM Inference

arXiv 2504.09561 ↗First posted 13 Apr 2025cs.ARcited by 0

On the map: Efficient LLM inference and serving → Serving systems and request scheduling

Abstract (arXiv)

In this paper, we propose LoopLynx, a scalable dataflow architecture for efficient LLM inference that optimizes FPGA usage through a hybrid spatial-temporal design. The design of LoopLynx incorporates a hybrid temporal-spatial architecture, where computationally intensive operators are implemented as large dataflow kernels. This achieves high throughput similar to spatial architecture, and organizing and reusing these kernels in a temporal way together enhances FPGA peak performance. Furthermore, to overcome the resource limitations of a single device, we provide a multi-FPGA distributed architecture that overlaps and hides all data transfers so that the distributed accelerators are fully utilized. By doing so, LoopLynx can be effectively scaled to multiple devices to further explore model parallelism for large-scale LLM inference. Evaluation of GPT-2 model demonstrates that LoopLynx can achieve comparable performance to state-of-the-art single FPGA-based accelerations. In addition, compared to Nvidia A100, our accelerator with a dual-FPGA configuration delivers a 2.52x speed-up in inference latency while consuming only 48.1% of the energy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Serving systems and request scheduling, one of 12 directions in Efficient LLM inference and serving. Builds on 12 of its 19 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

12 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs8 Jan

Other references · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  5. 2022Efficiently Scaling Transformer Inferencecited by 529
  6. 2020FTRANS: Energy-Efficient Acceleration of Transformers using FPGAcited by 52
  7. 2022DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generationcited by 46
  8. 2023Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inferencecited by 29
  9. 2021Accelerating Framework of Transformer by Hardware Design and Model Compression Co-Optimizationcited by 27
  10. 2021Hardware Acceleration of Fully Quantized BERT for Efficient Natural Language Processingcited by 21
  11. 2021NPE: An FPGA-based Overlay Processor for Natural Language Processingcited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs
  2. 2025HPU: High-Bandwidth Processing Unit for Scalable, Cost-effective LLM Inference via GPU Co-processing
  3. 2026LLM Inference on IMC-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelism
  4. 2026Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads ↗
  5. 2026FlashAccel: Leveraging High-Bandwidth Flash (HBF) for High-Throughput LLM Inference
  6. 2026FLINT: Efficiently Leveraging High Bandwidth Flash for Capacity-Scalable LLM Inference Acceleration
  7. 2025T-SAR: A Full-Stack Co-design for CPU-Only Ternary LLM Inference via In-Place SIMD ALU Reorganization
  8. 2026MOCAP: Wafer-Scale-Chip-Oriented Memory-Orchestrated Chunked Pipelining Framework for Prefill-Only LLM Inference