mlsanityData to 14 Oct 2026

MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference

arXiv 2604.00529 ↗First posted 1 Apr 2026cs.LG cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Quantization-aware training (QAT) is typically performed for a single target numeric format, while practical deployments often need to choose numerical precision at inference time based on hardware support or runtime constraints. We study multi-format QAT, where a single model is trained to be robust across multiple quantization formats. We find that multi-format QAT can match single-format QAT at each target precision, yielding one model that performs well overall across different formats, even formats that were not seen during training. To enable practical deployment, we propose the Slice-and-Scale conversion procedure for both MXINT and MXFP that converts a high-precision representation into lower-precision formats without re-training. Building on this, we introduce a pipeline that (i) trains a model with multi-format QAT, (ii) stores a single anchor format checkpoint (MXINT8/MXFP8), and (iii) allows on-the-fly conversion to lower MXINT or MXFP formats at runtime with negligible-or no-additional accuracy degradation. Together, these components provide a practical path to elastic precision scaling and allow selecting the runtime format at inference time across diverse deployment targets.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 32 references that are in the map. First posted on arXiv 1 Apr 2026.

Builds on

28 of 32 references.

References · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  11. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  12. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
16 more
  1. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  2. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  3. 2019Learned Step Size Quantizationcited by 628
  4. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  5. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  6. 2023Microscaling Data Formats for Deep Learningcited by 152
  7. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  8. 2019AdaBits: Neural Network Quantization with Adaptive Bit-Widthscited by 64
  9. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 54
  10. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 36
  11. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selectioncited by 22
  12. 2024Post Training Quantization of Large Language Models with Microscaling Formatscited by 12
  13. 2025MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Servingcited by 11
  14. 2024Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMscited by 3
  15. 2025NestedFP: High-Performance, Memory-Efficient Dual-Precision Floating Point Support for LLMscited by 3
  16. 2025OTARo: Once Tuning for All Precisions toward Robust On-Device LLMscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Target-Centric Survey of Quantization-Aware Training
  2. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  3. 2026Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
  4. 2025Compute-Optimal Quantization-Aware Training
  5. 2026QATFactory: A Versatile, Deployment-Aligned Framework for Quantization-aware Training and Distillation of LLMs
  6. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  7. 2026Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction
  8. 2023RAND: Robustness Aware Norm Decay For Quantized Seq2seq Models