TeLLMe v2: An Efficient End-to-End Ternary LLM Prefill and Decode Accelerator with Table-Lookup Matmul on Edge FPGAs
Not on the map. Why?
Abstract (arXiv)
With the emergence of wearable devices and other embedded systems, deploying large language models (LLMs) on edge platforms has become an urgent need. However, this is challenging because of their high computational and memory demands. Although recent low-bit quantization methods (e.g., BitNet, DeepSeek) compress weights to as low as 1.58~bits with minimal accuracy loss, edge deployment is still constrained by limited on-chip resources, power budgets, and the often-neglected long latency of the prefill stage. We present TeLLMe, the first table-lookup-based ternary LLM accelerator for low-power edge FPGAs that fully supports both prefill and autoregressive decoding using 1.58-bit weights and 8-bit activations. TeLLMe incorporates several novel techniques, including (1) a table-lookup-based ternary matrix multiplication (TLMM) engine utilizing grouped activations and online precomputation for low resource utilization and high throughput; (2) a fine-grained analytic URAM-based weight buffer management scheme for efficient loading and compute engine access; (3) a streaming dataflow architecture that fuses floating-point element-wise operations with linear computations to hide latency; (4) a reversed-reordered prefill stage attention with fused attention operations for high memory efficiency; and (5) a resource-efficient specialized decoding stage attention. Under a 5~W power budget, TeLLMe delivers up to 25~tokens/s decoding throughput and 0.45--0.96~s time-to-first-token (TTFT) for 64--128 token prompts, marking a significant energy-efficiency advancement in LLM inference on edge FPGAs.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 31 references that are in the map. First posted on arXiv 3 Oct 2025.
Builds on
20 of 31 references.
References · 20
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
- 2024FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAscited by 60
- 2020LogicNets: Co-Designed Neural Networks and Circuits for Extreme-Throughput Applicationscited by 55
- 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 47
- 2019LUTNet: Rethinking Inference in FPGA Soft Logiccited by 27
- 2023Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Expertscited by 23
- 2024EdgeLLM: A Highly Efficient CPU-FPGA Heterogeneous Edge Accelerator for Large Language Modelscited by 19
- 2024Designing Efficient LLM Accelerators for Edge Devicescited by 13
- 2024LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAscited by 12
8 more
- 2025LightMamba: Efficient Mamba Acceleration on FPGA with Quantization and Hardware Co-designcited by 11
- 2022A Two-Stage Efficient 3-D CNN Framework for EEG Based Emotion Recognitioncited by 7
- 2024Table-Lookup MAC: Scalable Processing of Quantised Neural Networks in FPGA Soft Logiccited by 6
- 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAscited by 5
- 2025TerEffic: Highly Efficient Ternary LLM Inference on FPGAcited by 4
- 2025COBRA: Algorithm-Architecture Co-optimized Binary Transformer Accelerator for Edge Inferencecited by 4
- 2025TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edgecited by 4
- 2025MEADOW: Memory-efficient Dataflow and Data Packing for Low Power Edge LLMscited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.