mlsanityData to 7 Oct 2026

FlexLLM: Composable HLS Library for Flexible Hybrid LLM Accelerator Design

arXiv 2601.15710 ↗First posted 22 Jan 2026cs.AR cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

We present FlexLLM, a composable High-Level Synthesis (HLS) library for rapid development of domain-specific LLM accelerators. FlexLLM exposes key architectural degrees of freedom for stage-customized inference, enabling hybrid designs that tailor temporal reuse and spatial dataflow differently for prefill and decode, and provides a comprehensive quantization suite to support accurate low-bit deployment. Using FlexLLM, we build a complete inference system for the Llama-3.2 1B model in under two months with only 1K lines of code. The system includes: (1) a stage-customized accelerator with hardware-efficient quantization (12.68 WikiText-2 PPL) surpassing SpinQuant baseline, and (2) a Hierarchical Memory Transformer (HMT) plug-in for efficient long-context processing. On the AMD U280 FPGA at 16nm, the accelerator achieves 1.29× end-to-end speedup, 1.64× higher decode throughput, and 3.14× better energy efficiency than an NVIDIA A100 GPU (7nm) running BF16 inference; projected results on the V80 FPGA at 7nm reach 4.71×, 6.55×, and 4.13×, respectively. In long-context scenarios, integrating the HMT plug-in reduces prefill latency by 23.23× and extends the context window by 64×, delivering 1.10×/4.86× lower end-to-end latency and 5.21×/6.27× higher energy efficiency on the U280/V80 compared to the A100 baseline. FlexLLM thus bridges algorithmic innovation in LLM inference and high-performance accelerators with minimal manual effort.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

25 of 38 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2024The Llama 3 Herd of Modelscited by 14,192
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  10. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  12. 2024SpinQuant: LLM quantization with learned rotationscited by 446
13 more
  1. 2023A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writingcited by 98
  2. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 65
  3. 2024FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAscited by 60
  4. 2024MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Modelscited by 48
  5. 2022DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generationcited by 46
  6. 2023Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inferencecited by 29
  7. 2024HMT: Hierarchical Memory Transformer for Efficient Long Context Language Processingcited by 20
  8. 2024EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Modelscited by 19
  9. 2024Allo: A Programming Model for Composable Accelerator Designcited by 16
  10. 2024SSR: Spatial Sequential Hybrid Architecture for Latency Throughput Tradeoff in Transformer Accelerationcited by 9
  11. 2022TAPA: A Scalable Task-Parallel Dataflow Programming Framework for Modern FPGAs with Co-Optimization of HLS and Physical Designcited by 5
  12. 2025InTAR: Inter-Task Auto-Reconfigurable Accelerator Design for High Data Volume Variation in DNNscited by 3
  13. 2025StreamTensor: Make Tensors Stream in Dataflow Accelerators for LLMscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026A Persistent-State Dataflow Accelerator for Memory-Bound Linear Attention Decode on FPGAcited by 2
  2. 2026Understand and Accelerate Memory Processing Pipeline for Large Language Model Inferencecited by 0
  3. 2026Can Agents Design Better Chips with a Higher Level Abstraction?cited by 0