mlsanityData to 14 Oct 2026

REx: Data-Free Residual Quantization Error Expansion

arXiv 2203.14645 ↗First posted 28 Mar 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) are ubiquitous in computer vision and natural language processing, but suffer from high inference cost. This problem can be addressed by quantization, which consists in converting floating point operations into a lower bit-width format. With the growing concerns on privacy rights, we focus our efforts on data-free methods. However, such techniques suffer from their lack of adaptability to the target devices, as a hardware typically only support specific bit widths. Thus, to adapt to a variety of devices, a quantization method shall be flexible enough to find good accuracy v.s. speed trade-offs for every bit width and target device. To achieve this, we propose REx, a quantization method that leverages residual error expansion, along with group sparsity and an ensemble approximation for better parallelization. REx is backed off by strong theoretical guarantees and achieves superior performance on every benchmarked application (from vision to NLP tasks), architecture (ConvNets, transformers) and bit-width (from int8 to ternary quantization).

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 47 of its 77 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

47 of 77 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  9. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  10. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  12. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
35 more
  1. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  2. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  3. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  7. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  8. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  10. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  11. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  12. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  13. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  14. 2014Compressing Deep Convolutional Networks using Vector Quantizationcited by 195
  15. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  16. 2015Compressing Neural Networks with the Hashing Trickcited by 150
  17. 2015Fixed Point Quantization of Deep Convolutional Networkscited by 143
  18. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 99
  19. 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
  20. 2020Generative Low-bitwidth Data Free Quantizationcited by 88
  21. 2018Training and Inference with Integers in Deep Neural Networkscited by 86
  22. 2021Diversifying Sample Generation for Accurate Data-Free Quantizationcited by 63
  23. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 61
  24. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 55
  25. 2021PokeBNN: A Binary Pursuit of Lightweight Accuracycited by 49
  26. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48
  27. 2019Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorizationcited by 45
  28. 2021IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantizationcited by 43
  29. 2018Binary Ensemble Neural Network: More Bits per Network or More Networks per Bit?cited by 39
  30. 2020MixMix: All You Need for Data-Free Compression Are Feature and Data Mixingcited by 29
  31. 2018A Quantization-Friendly Separable Convolution for MobileNetscited by 22
  32. 2021RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networkscited by 19
  33. 2019Equi-normalization of Neural Networkscited by 16
  34. 2021APNN-TC: Accelerating Arbitrary Precision Neural Networks on Ampere GPU Tensor Corescited by 12
  35. 2015Deep CNN Ensemble with Data Augmentation for Object Detectioncited by 10

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 2.

6 papers.

  1. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  2. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  3. 2024Skipping Computations in Multimodal LLMscited by 7
  4. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 7
  5. 2023Designing strong baselines for ternary neural network quantization through support and mass equalizationcited by 0
  6. 2023PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization
  2. 2023PLUM: Improving Inference Efficiency By Leveraging Repetition-Sparsity Trade-Off
  3. 2024Reclaiming Residual Knowledge: A Novel Paradigm to Low-Bit Quantization
  4. 2024Minimize Quantization Output Error with Bias Compensation
  5. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classification
  6. 2024Effective Interplay between Sparsity and Quantization: From Theory to Practice
  7. 2024A Data-Free Analytical Quantization Scheme for Deep Learning Models
  8. 2024COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization