mlsanityData to 14 Oct 2026

LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs

arXiv 2409.11424 ↗First posted 12 Sep 2024cs.ARcited by 12

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable abilities in natural language processing. However, their deployment on resource-constrained embedded devices remains difficult due to memory and computational demands. In this paper, we present an FPGA-based accelerator designed to improve LLM inference performance on embedded FPGAs. We employ post-training quantization to reduce model size and optimize for off-chip memory bandwidth. Our design features asynchronous computation and a fully pipelined accelerator for matrix-vector multiplication. Experiments of the TinyLlama 1.1B model on a Xilinx ZCU102 platform show a 14.3-15.8x speedup and a 6.1x power efficiency improvement over running exclusively on ZCU102 processing system (PS).

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Sep 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 1.

12 papers.

  1. 2024Mitigating the Risk of Health Inequity Exacerbated by Large Language Modelscited by 21
  2. 2024HADES: Hardware Accelerated Decoding for Efficient Speculation in Large Language Modelscited by 15
  3. 2024LoRA-LiteE: A Computationally Efficient Framework for Chatbot Preference-Tuningcited by 10
  4. 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAscited by 10
  5. 2025Pushing up to the Limit of Memory Bandwidth and Capacity Utilization for Efficient LLM Decoding on Embedded FPGAcited by 7
  6. 2025On-Device Qwen2.5: Efficient LLM Inference with Model Compression and Hardware Accelerationcited by 6
  7. 2025MEADOW: Memory-efficient Dataflow and Data Packing for Low Power Edge LLMscited by 4
  8. 2025TeLLMe v2: An Efficient End-to-End Ternary LLM Prefill and Decode Accelerator with Table-Lookup Matmul on Edge FPGAscited by 2
  9. 2025NSFlow: An End-to-End FPGA Framework with Scalable Dataflow Architecture for Neuro-Symbolic AIcited by 1
  10. 2025AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Designcited by 1
  11. 2025PD-Swap: Prefill-Decode Logic Swapping for End-to-End LLM Inference on Edge FPGAs via Dynamic Partial Reconfigurationcited by 1
  12. 2026LLMscope: Extracting LLM Assets from Edge AI Chips via Optical Probingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs
  2. 2025FPGA Co-Design for Efficient N:M Sparse and Quantized Model Inference
  3. 2025TerEffic: Highly Efficient Ternary LLM Inference on FPGA
  4. 2024Designing Efficient LLM Accelerators for Edge Devices
  5. 2026AI Hardware Accelerators for Large Language Models: Architectures and the Memory Wall ↗
  6. 2026Understand and Accelerate Memory Processing Pipeline for Large Language Model Inference
  7. 2023Efficient LLM Inference on CPUs
  8. 2024New Solutions on LLM Acceleration, Optimization, and Application