mlsanityData to 14 Oct 2026

NestQuant: Post-Training Integer-Nesting Quantization for On-Device DNN

arXiv 2506.17870 ↗First posted 22 Jun 2025cs.LG cs.AI cs.DCcited by 4

Not on the map. Why?

Abstract (arXiv)

Deploying quantized deep neural network (DNN) models with resource adaptation capabilities on ubiquitous Internet of Things (IoT) devices to provide high-quality AI services can leverage the benefits of compression and meet multi-scenario resource requirements. However, existing dynamic/mixed precision quantization requires retraining or special hardware, whereas post-training quantization (PTQ) has two limitations for resource adaptation: (i) The state-of-the-art PTQ methods only provide one fixed bitwidth model, which makes it challenging to adapt to the dynamic resources of IoT devices; (ii) Deploying multiple PTQ models with diverse bitwidths consumes large storage resources and switching overheads. To this end, this paper introduces a resource-friendly post-training integer-nesting quantization, i.e., NestQuant, for on-device quantized model switching on IoT devices. The proposed NestQuant incorporates the integer weight decomposition, which bit-wise splits quantized weights into higher-bit and lower-bit weights of integer data types. It also contains a decomposed weights nesting mechanism to optimize the higher-bit weights by adaptive rounding and nest them into the original quantized weights. In deployment, we can send and store only one NestQuant model and switch between the full-bit/part-bit model by paging in/out lower-bit weights to adapt to resource changes and reduce consumption. Experimental results on the ImageNet-1K pretrained DNNs demonstrated that the NestQuant model can achieve high performance in top-1 accuracy, and reduce in terms of data transmission, storage consumption, and switching overheads. In particular, the ResNet-101 with INT8 nesting INT6 can achieve 78.1% and 77.9% accuracy for full-bit and part-bit models, respectively, and reduce switching overheads by approximately 78.1% compared with diverse bitwidths PTQ models.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 22 Jun 2025.

Builds on

27 of 53 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  12. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
15 more
  1. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  2. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  3. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  4. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  5. 2020SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloudcited by 112
  6. 2018NestDNN: Resource-Aware Multi-Tenant On-Device Deep Learning for Continuous Mobile Visioncited by 68
  7. 2022Deep Differentiable Logic Gate Networkscited by 67
  8. 2019AdaBits: Neural Network Quantization with Adaptive Bit-Widthscited by 64
  9. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 55
  10. 2018A First Look at Deep Learning Apps on Smartphonescited by 47
  11. 2019Any-Precision Deep Neural Networkscited by 40
  12. 2022LightFR: Lightweight Federated Recommendation with Privacy-preserving Matrix Factorizationcited by 26
  13. 2020Energy Efficient Federated Learning over Heterogeneous Mobile Devices via Joint Design of Weight Quantization and Wireless Transmissioncited by 23
  14. 2023AdaptiveNet: Post-deployment Neural Architecture Adaptation for Diverse Edge Environmentscited by 19
  15. 2023EQ-Net: Elastic Quantization Neural Networkscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026MatGPTQ: Efficient and Accurate Inference over Nested Quantized Modelscited by 3
  2. 2026SimCert: Probabilistic Certification for Behavioral Similarity in Deep Neural Network Compressioncited by 1
  3. 2025OptiFLIDS: Optimized Federated Learning for Energy-Efficient Intrusion Detection in IoTcited by 0
  4. 2026AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025DQT: Dynamic Quantization Training via Dequantization-Free Nested Integer Arithmetic
  2. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMs
  3. 2026ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
  4. 2026A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPU
  5. 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs
  6. 2024DNN Memory Footprint Reduction via Post-Training Intra-Layer Multi-Precision Quantization
  7. 2026NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models
  8. 2022On-Device Training Under 256KB Memory