mlsanityData to 14 Oct 2026

ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs

arXiv 2602.17698 ↗First posted 6 Feb 2026cs.LG cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Post-training weight quantization is crucial for reducing the memory and inference cost of large language models (LLMs), yet pushing the average precision below 4 bits remains challenging due to highly non-uniform weight sensitivity and the lack of principled precision allocation. Existing solutions use irregular fine-grained mixed-precision with high runtime overhead or rely on heuristics or highly constrained precision allocation strategies. In this work, we propose ScaleBITS, a mixed-precision quantization framework that enables automated, fine-grained bitwidth allocation under a memory budget while preserving hardware efficiency. Guided by a new sensitivity analysis, we introduce a hardware-aligned, block-wise weight partitioning scheme, powered by bi-directional channel reordering. We formulate global bitwidth allocation as a constrained optimization problem and develop a scalable approximation to the greedy algorithm, enabling end-to-end principled allocation. Experiments show that ScaleBITS significantly improves over uniform-precision quantization (up to +36%) and outperforms state-of-the-art sensitivity-aware baselines (up to +13%) in ultra-low-bit regime, without adding runtime overhead.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 43 of its 53 references that are in the map. First posted on arXiv 6 Feb 2026.

Builds on

43 of 53 references.

References · 43

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  9. 2021Program Synthesis with Large Language Modelscited by 3,637
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
31 more
  1. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  5. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  6. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  7. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  8. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  9. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  10. 2024Massive Activations in Large Language Modelscited by 294
  11. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  12. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  13. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 182
  14. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  15. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  16. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  17. 2020Rethinking Differentiable Search for Mixed-Precision Neural Networkscited by 82
  18. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  19. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  20. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 54
  21. 2024The Super Weight in Large Language Modelscited by 52
  22. 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Modelscited by 34
  23. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 32
  24. 2024Progressive Mixed-Precision Decoding for Efficient LLM Inferencecited by 18
  25. 2023Fast and Efficient 2-bit LLM Inference on GPU: 2/4/16-bit in a Weight Matrix with Asynchronous Dequantizationcited by 14
  26. 2025AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Modelscited by 14
  27. 2025BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inferencecited by 11
  28. 2025FGMP: Fine-Grained Mixed-Precision Weight and Activation Quantization for Hardware-Accelerated LLM Inferencecited by 9
  29. 2025ICQuant: Index Coding enables Low-bit LLM Quantizationcited by 9
  30. 2025Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compressioncited by 7
  31. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Q-Strata: Hierarchical Bit Allocation for Mixed-Precision Quantization of Mixture-of-Experts LLMscited by 3
  2. 2026Saturation Makes Quantization Error Additive: A Coverage Model with a Certificatecited by 0
  3. 2026AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Channel-Wise Mixed-Precision Quantization for Large Language Models
  2. 2025AnyBCQ: Hardware Efficient Flexible Binary-Coded Quantization for Multi-Precision LLMs
  3. 2025FineQ: Software-Hardware Co-Design for Low-Bit Fine-Grained Mixed-Precision Quantization of LLMs
  4. 2026SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Models
  5. 2024Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques
  6. 2025FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization
  7. 2025Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
  8. 2026RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference