mlsanityData to 14 Oct 2026

OMPQ: Orthogonal Mixed Precision Quantization

arXiv 2109.07865 ↗First posted 16 Sep 2021cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

To bridge the ever increasing gap between deep neural networks' complexity and hardware capability, network quantization has attracted more and more research attention. The latest trend of mixed precision quantization takes advantage of hardware's multiple bit-width arithmetic operations to unleash the full potential of network quantization. However, this also results in a difficult integer programming formulation, and forces most existing approaches to use an extremely time-consuming search process even with various relaxations. Instead of solving a problem of the original integer programming, we propose to optimize a proxy metric, the concept of network orthogonality, which is highly correlated with the loss of the integer programming but also easy to optimize with linear programming. This approach reduces the search time and required data amount by orders of magnitude, with little compromise on quantization accuracy. Specifically, we achieve 72.08% Top-1 accuracy on ResNet-18 with 6.7Mb, which does not require any searching iterations. Given the high efficiency and low data dependency of our algorithm, we used it for the post-training quantization, which achieve 71.27% Top-1 accuracy on MobileNetV2 with only 1.5Mb. Our code is available at https://github.com/MAC-AutoML/OMPQ.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 37 of its 58 references that are in the map. First posted on arXiv 16 Sep 2021.

Builds on

37 of 58 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2014Going Deeper with Convolutionscited by 4,241
  7. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  10. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  12. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
25 more
  1. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  2. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  3. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  4. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  5. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  6. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  7. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  8. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  9. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  10. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  11. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  12. 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 235
  13. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
  14. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 117
  15. 2020FracBits: Mixed Precision Quantization via Fractional Bit-Widthscited by 55
  16. 2018Value-aware Quantization for Training and Inference of Neural Networkscited by 52
  17. 2020PAMS: Quantized Super-Resolution via Parameterized Max Scalecited by 48
  18. 2019Circulant Binary Convolutional Networks: Enhancing the Performance of 1-bit DCNNs with Circulant Back Propagationcited by 28
  19. 2020BinaryDuo: Reducing Gradient Mismatch in Binary Activation Network by Coupling Binary Activationscited by 23
  20. 2019Multinomial Distribution Learning for Effective Neural Architecture Searchcited by 17
  21. 2021An Information Theory-inspired Strategy for Automatic Network Pruningcited by 17
  22. 2021You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradientcited by 14
  23. 2019DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruningcited by 13
  24. 2020One Weight Bitwidth to Rule Them Allcited by 13
  25. 2020Rethinking Performance Estimation in Neural Architecture Searchcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 5; 2025: 9; 2026: 13.

39 papers.

Sort:

By area:

  1. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  2. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 28
  3. 2022SDQ: Stochastic Differentiable Quantization with Mixed Precisioncited by 28
  4. 2023A Practical Mixed Precision Algorithm for Post-Training Quantizationcited by 24
  5. 2023EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantizationcited by 22
  6. 2023Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspectivecited by 17
  7. 2025Quantized Visual Geometry Grounded Transformercited by 17
  8. 2023Patch-wise Mixed-Precision Quantization of Vision Transformercited by 15
  9. 2026SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Modelscited by 13
  10. 2025MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillationcited by 12
  11. 2023Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devicescited by 11
  12. 2024Channel-Wise Mixed-Precision Quantization for Large Language Modelscited by 11
27 more
  1. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  2. 2026DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantizationcited by 8
  3. 2026ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Modelscited by 8
  4. 2023Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programmingcited by 7
  5. 2026Motion-Aware Caching for Efficient Autoregressive Video Generationcited by 6
  6. 2024ParZC: Parametric Zero-Cost Proxies for Efficient NAScited by 5
  7. 2023Towards Optimal Compression: Joint Pruning and Quantizationcited by 4
  8. 2023Lossy and Lossless (L2) Post-training Model Size Compressioncited by 4
  9. 2025InfoQ: Mixed-Precision Quantization via Global Information Flowcited by 4
  10. 2023Efficient and Effective Methods for Mixed Precision Neural Network Quantization for Faster, Energy-efficient Inferencecited by 3
  11. 2025MixA-Q: Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspectivecited by 3
  12. 2023On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networkscited by 1
  13. 2024AdaQAT: Adaptive Bit-Width Quantization-Aware Trainingcited by 1
  14. 2025Where and How to Enhance: Discovering Bit-Width Contribution for Mixed Precision Quantizationcited by 1
  15. 2026Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoningcited by 1
  16. 2026OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Modelscited by 1
  17. 2026Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinementcited by 1
  18. 2024A Data-Free Analytical Quantization Scheme for Deep Learning Modelscited by 0
  19. 2025Nearly Lossless Adaptive Bit Switchingcited by 0
  20. 2025QuantFace: Efficient Quantization for Face Restorationcited by 0
  21. 2025Towards Training-free Automatic Proxy Discovery via Large Language Models for Mixed Precision Quantizationcited by 0
  22. 2026A2RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generationcited by 0
  23. 2026FUSE: Frequency-domain Unification and Spectral Energy Alignment for Multi-modal Object Re-Identificationcited by 0
  24. 2026CommitKV: Lifecycle-Aware KV Cache Compression via Commit Transitions for Multi-Turn Agentscited by 0
  25. 2026UC-VLM: Consistency-Driven Learning for AI-Generated Image Detection with Vision-Language Large Modelscited by 0
  26. 2026STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantizationcited by 0
  27. 2026Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimization
  2. 2023Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge
  3. 2023MixQuant: Mixed Precision Quantization with a Bit-width Optimization Search
  4. 2021Bit-Mixer: Mixed-precision networks with runtime bit-width selection
  5. 2025Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inference
  6. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance
  7. 2025Efficient Mixed Precision Quantization in Graph Neural Networks
  8. 2022FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation