mlsanityData to 14 Oct 2026

StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMs

arXiv 2610.05977 ↗First posted 5 Oct 2026cs.LG cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Serving a large language model (LLM) across a fleet of deployments requires several weight-precision operating points. Multi-precision formats serve them all from one stream whose prefixes are valid lower-precision codes, instead of storing multiple copies. We present StagQ, a multi-precision weight format whose main stream is a 2-bit group-wise affine base followed by a configurable number of 1-bit refinement planes on a dyadic step schedule. Every supported precision is a readable prefix, decoded by an affine map derived from metadata shared across all precisions, with no per-weight lookup. A sparse side record, filled both before and after the grid is fitted, holds out the few weights the grid serves worst. We report two configurations of the encoder. At two bits the cheaper one leads the strongest multi-precision baseline on Llama-3.1-8B, Phi-4, and OLMo-2-7B by 3.1 to 7.0 MMLU points, at a slightly lower logical rate. At three bits it leads on Llama-3.1-8B, leads on Phi-4 at a higher rate, and ties on OLMo-2-7B. At four bits it ties on all three, at a higher rate. In a batch-one matrix-vector product on an NVIDIA A100 GPU, timed on synthetic weights, our kernel is faster than the two baseline kernels in most shape-precision cases.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 27 references that are in the map. First posted on arXiv 5 Oct 2026.

Builds on

15 of 27 references.

References · 15

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2016Pointer Sentinel Mixture Modelscited by 3,096
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  7. 2024Phi-4 Technical Reportcited by 711
  8. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  9. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  10. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 54
  11. 2025D2MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Servingcited by 20
  12. 2025AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMscited by 9
3 more
  1. 2025SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inferencecited by 3
  2. 2026Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMscited by 3
  3. 2026BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models
  2. 2025FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs
  3. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs
  4. 2024Channel-Wise Mixed-Precision Quantization for Large Language Models
  5. 2026BARQ: Balanced Codebook Refinement for Low-Bit LLM Quantization
  6. 2024Progressive Mixed-Precision Decoding for Efficient LLM Inference
  7. 2026SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Models
  8. 2025FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization