mlsanityData to 14 Oct 2026

FlexPosit: Tunable Fractional Precision for LLM Inference Accelerators

arXiv 2609.04724 ↗First posted 4 Sep 2026cs.ARcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) offer remarkable capabilities but impose prohibitive compute and energy costs. Quantization governs the trade-offs between accuracy and hardware efficiency across granularity and bit-width. Finer granularity (e.g., group-wise) provides high accuracy but incurs scaling and control overhead, while coarser granularity (e.g., channel-wise) has lower overhead but loses accuracy at low precision. Meanwhile, mixed-precision quantization exposes rich accuracy-efficiency trade-offs algorithmically, but existing LLM accelerators remain limited to discrete precision modes, leaving the fractional design space between them unexplored. FlexPosit bridges these gaps through co-design of Posit-based quantization and a precision-tunable bit-serial architecture. Algorithmically, FlexPosit employs distribution-aware quantization with hardware-aligned, sensitivity-guided mixed-precision allocation, leveraging the Posit format's tapered precision to achieve group-wise-like accuracy with channel-wise-like regularity. Architecturally, FlexPosit is a unified bit-serial systolic array with lightweight per-column decoders, unified Processing Elements (PEs), and a global precision controller, enabling tunable fractional precision while preserving fully regular systolic dataflow. Across diverse LLMs, FlexPosit achieves near-FP16 accuracy with sub-5-bit fractional weights. It achieves 1.8x higher throughput and 1.2x lower energy than BitMoD (group-wise quantization), and 1.5x higher throughput and 2.0x lower energy than OliVe (channel-wise quantization), establishing a new Pareto frontier for precision-tunable LLM acceleration.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

28 of 53 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2023Mistral 7Bcited by 3,164
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  10. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  11. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  12. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
16 more
  1. 2023Microscaling Data Formats for Deep Learningcited by 152
  2. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
  3. 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 106
  4. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 92
  5. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  6. 2023Training Transformers with 4-bit Integerscited by 60
  7. 2025Optimizing Large Language Model Training Using FP4 Quantizationcited by 56
  8. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 51
  9. 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Modelscited by 34
  10. 2023Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulatorcited by 30
  11. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 28
  12. 2024What Makes Quantization for Large Language Models Hard? An Empirical Study from the Lens of Perturbationcited by 23
  13. 2024BitMoD: Bit-serial Mixture-of-Datatype LLM Accelerationcited by 18
  14. 2022FIT: A Metric for Model Sensitivitycited by 11
  15. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 5
  16. 2021The Accuracy and Efficiency of Posit Arithmeticcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization
  2. 2025FGMP: Fine-Grained Mixed-Precision Weight and Activation Quantization for Hardware-Accelerated LLM Inference
  3. 2024MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
  4. 2025FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs
  5. 2026Hardware Generation and Exploration of Lookup Table-Based Accelerators for 1.58-bit LLM Inference
  6. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  7. 2026FlexLLM: Composable HLS Library for Flexible Hybrid LLM Accelerator Design
  8. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs