mlsanityData to 14 Oct 2026

AMQ: Enabling AutoML for Mixed-precision Weight-Only Quantization of Large Language Models

arXiv 2509.12019 ↗First posted 15 Sep 2025cs.LG cs.AI cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

To enable broader deployment of Large Language Models (LLMs), it is essential to identify the best-performing model under strict memory constraints. We present AMQ, Automated Mixed-Precision Weight-Only Quantization, a framework that assigns layer-wise quantization bit-widths to optimally balance model quality and memory usage. However, the combinatorial search space, with over 10^{100} possible configurations, makes conventional black-box optimization infeasible. AMQ overcomes this challenge through four key innovations:(1) search space pruning using prior knowledge to exclude unpromising configurations, (2) quantization proxy to bypass costly format conversions during search, (3) quality predictor to minimize evaluation overhead, and (4) iterative search-and-update strategy for fast and stable convergence. By integrating these components, AMQ efficiently explores the quality-efficiency landscape, reaching the Pareto frontier and yielding LLMs that are both compact and high-performing. Our code is available at https://github.com/dlwns147/amq.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 24 of its 42 references that are in the map. First posted on arXiv 15 Sep 2025.

Builds on

24 of 42 references.

References · 24

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  11. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  12. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
12 more
  1. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  2. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  3. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  4. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  5. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  6. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  7. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  8. 2019Neural Predictor for Neural Architecture Searchcited by 118
  9. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  10. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkitcited by 33
  11. 2019Multi-Objective Reinforced Evolution in Mobile Neural Architecture Searchcited by 24
  12. 2024QEFT: Quantization for Efficient Fine-Tuning of LLMscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 5
  2. 2026AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuningcited by 5
  3. 2025TreeQ: Pushing the Quantization Boundary of Diffusion Transformer via Tree-Structured Mixed-Precision Searchcited by 3
  4. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMscited by 3
  5. 2026KronQ: LLM Quantization via Kronecker-Factored Hessiancited by 3
  6. 2026EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecturecited by 2
  7. 2026MixQuant: Adaptive Mixed-Precision Quantization for Large Language Modelscited by 2
  8. 2026SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Modelscited by 1
  9. 2025SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMscited by 0
  10. 2026Saturation Makes Quantization Error Additive: A Coverage Model with a Certificatecited by 0
  11. 2026PolyQ: Codesigning End-to-End Quantization Framework for Scalable Edge CPU LLM Inferencecited by 0
  12. 2026ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMMcited by 0
2 more
  1. 2026RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Modelscited by 0
  2. 2026Predicting Quantization Price for Selecting PTQ Configurations Before Deploymentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Models
  2. 2024Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models
  3. 2024Channel-Wise Mixed-Precision Quantization for Large Language Models
  4. 2025SiLQ: Simple Large Language Model Quantization-Aware Training
  5. 2025Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models
  6. 2024Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
  7. 2024Post Training Quantization of Large Language Models with Microscaling Formats
  8. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Models