mlsanityData to 14 Oct 2026

Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Training

arXiv 2605.25054 ↗First posted 24 May 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Deploying deep neural networks on resource-constrained 6G edge devices demands aggressive compression with minimal accuracy loss. Quantization-Aware Training (QAT) has emerged as a leading compression approach; however, existing mixed-precision methods typically operate at coarse layer- or channel-level granularity. These methods often rely on heuristic or search-based bit-allocation strategies, which may overlook fine-grained variability at the neuron level. We propose Neuron-Level Mixed-Precision QAT (NMP-QAT), where each neuron independently learns its own discrete precision during training. Starting from low-bit precision, NMP-QAT expands bit-width only when training signals demand it, via differentiable surrogates and straight-through estimators, while preserving a fully discrete inference graph. This adaptability extends to both weights and activations, reducing memory movement. Evaluated on telecom and non-telecom datasets across MLP and tabular foundation model architectures, NMP-QAT achieves superior compression-accuracy trade-offs over mixed-precision QAT baselines, making it well-suited for Green AI deployments at the network edge.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 17 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

12 of 17 references.

References · 12

  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
  5. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  6. 2020Tabular Transformers for Modeling Multivariate Time Seriescited by 77
  7. 2023Green Edge AI: A Contemporary Surveycited by 25
  8. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
  9. 2018ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networkscited by 14
  10. 2024Channel-Wise Mixed-Precision Quantization for Large Language Modelscited by 11
  11. 2020Learning Quantized Neural Nets by Coarse Gradient Method for Non-linear Classificationcited by 8
  12. 2024Mixed-Precision Quantization: Make the Best Use of Bits Where They Matter Mostcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023A Practical Mixed Precision Algorithm for Post-Training Quantization
  2. 2025Compute-Optimal Quantization-Aware Training
  3. 2023Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge
  4. 2026Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
  5. 2024AdaQAT: Adaptive Bit-Width Quantization-Aware Training
  6. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization
  7. 2022Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes
  8. 2022Edge Inference with Fully Differentiable Quantized Mixed Precision Neural Networks