mlsanityData to 14 Oct 2026

T-SAR: A Full-Stack Co-design for CPU-Only Ternary LLM Inference via In-Place SIMD ALU Reorganization

arXiv 2511.13676 ↗First posted 17 Nov 2025cs.AR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent advances in LLMs have outpaced the computational and memory capacities of edge platforms that primarily employ CPUs, thereby challenging efficient and scalable deployment. While ternary quantization enables significant resource savings, existing CPU solutions rely heavily on memory-based lookup tables (LUTs) which limit scalability, and FPGA or GPU accelerators remain impractical for edge use. This paper presents T-SAR, the first framework to achieve scalable ternary LLM inference on CPUs by repurposing the SIMD register file for dynamic, in-register LUT generation with minimal hardware modifications. T-SAR eliminates memory bottlenecks and maximizes data-level parallelism, delivering 5.6-24.5x and 1.1-86.2x improvements in GEMM latency and GEMV throughput, respectively, with only 3.2% power and 1.4% area overheads in SIMD units. T-SAR achieves up to 2.5-4.9x the energy efficiency of an NVIDIA Jetson AGX Orin, establishing a practical approach for efficient LLM inference on edge platforms.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 23 of its 41 references that are in the map. First posted on arXiv 17 Nov 2025.

Builds on

23 of 41 references.

References · 23

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  3. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  4. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  5. 2024Large Language Models: A Surveycited by 525
  6. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  7. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
  8. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  9. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 100
  10. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  11. 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 79
  12. 2022BiBERT: Accurate Fully Binarized BERTcited by 73
11 more
  1. 2024LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inferencecited by 70
  2. 2024T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edgecited by 55
  3. 2025Bitnet.cpp: Efficient Edge Inference for Ternary LLMscited by 22
  4. 20241-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUscited by 12
  5. 2025TeLLMe: An Energy-Efficient Ternary LLM Accelerator for Prefilling and Decoding on Edge FPGAscited by 10
  6. 2025TerEffic: Highly Efficient Ternary LLM Inference on FPGAcited by 6
  7. 2024Efficiently Distilling LLMs for Edge Applicationscited by 5
  8. 2024Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operationscited by 5
  9. 2025LVLM_CSP: Accelerating Large Vision Language Models via Clustering, Scattering, and Pruning for Reasoning Segmentationcited by 4
  10. 2023SIMD Everywhere Optimization from ARM NEON to RISC-V Vector Extensionscited by 3
  11. 2025AraXL: A Physically Scalable, Ultra-Wide RISC-V Vector Processor Design for Fast and Efficient Computation on Long Vectorscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025AVERY: Intent-Driven Adaptive VLM Split Computing via Embodied Self-Awareness for Efficient Disaster Response Systemscited by 4
  2. 2025Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devicescited by 2
  3. 2026PolyQ: Codesigning End-to-End Quantization Framework for Scalable Edge CPU LLM Inferencecited by 0
  4. 2026ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMMcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
  2. 2026VitaLLM: A Versatile, Ultra-Compact Ternary LLM Accelerator with Dependency-Aware Scheduling ↗
  3. 2025TENET: An Efficient Sparsity-Aware LUT-Centric Architecture for Ternary LLM Inference On Edge
  4. 2026Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levels
  5. 2026Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads ↗
  6. 2026TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs
  7. 2026Architecture-Aware LLM Inference Optimization on AMD Instinct GPUs: A Comprehensive Benchmark and Deployment Study
  8. 2026PALUTE: Processing-In-Memory Acceleration via Lookup Table for Edge LLM Inference