mlsanityData to 14 Oct 2026

PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers

arXiv 2209.05687 ↗First posted 13 Sep 2022cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Data-free quantization can potentially address data privacy and security concerns in model compression, and thus has been widely investigated. Recently, PSAQ-ViT designs a relative value metric, patch similarity, to generate data from pre-trained vision transformers (ViTs), achieving the first attempt at data-free quantization for ViTs. In this paper, we propose PSAQ-ViT V2, a more accurate and general data-free quantization framework for ViTs, built on top of PSAQ-ViT. More specifically, following the patch similarity metric in PSAQ-ViT, we introduce an adaptive teacher-student strategy, which facilitates the constant cyclic evolution of the generated samples and the quantized model (student) in a competitive and interactive fashion under the supervision of the full-precision model (teacher), thus significantly improving the accuracy of the quantized model. Moreover, without the auxiliary category guidance, we employ the task- and model-independent prior information, making the general-purpose scheme compatible with a broad range of vision tasks and models. Extensive experiments are conducted on various models on image classification, object detection, and semantic segmentation tasks, and PSAQ-ViT V2, with the naive quantization strategy and without access to real-world data, consistently achieves competitive results, showing potential as a powerful baseline on data-free quantization for ViTs. For instance, with Swin-S as the (backbone) model, 8-bit quantization reaches 82.13 top-1 accuracy on ImageNet, 50.9 box AP and 44.1 mask AP on COCO, and 47.2 mIoU on ADE20K. We hope that accurate and general PSAQ-ViT V2 can serve as a potential and practice solution in real-world applications involving sensitive data. Code is released and merged at: https://github.com/zkkli/PSAQ-ViT.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 54 of its 70 references that are in the map. First posted on arXiv 13 Sep 2022.

Builds on

54 of 70 references.

References · 54

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  10. 2021ViViT: A Video Vision Transformercited by 1,620
  11. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  12. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
38 more
  1. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  2. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  3. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  4. 2021Transformers in Vision: A Surveycited by 1,013
  5. 2020Pre-Trained Image Processing Transformercited by 893
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  7. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  8. 2020A Survey on Visual Transformercited by 654
  9. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  10. 2021Transformer in Transformercited by 633
  11. 2019Learned Step Size Quantizationcited by 628
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  13. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  14. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  15. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  16. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  17. 2021Transformer Trackingcited by 412
  18. 2021DeepViT: Towards Deeper Vision Transformercited by 289
  19. 2021Post-Training Quantization for Vision Transformercited by 278
  20. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  21. 2021Video Transformer Networkcited by 272
  22. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  23. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 234
  24. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 207
  25. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  26. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  27. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  28. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  29. 2021Patch Slimming for Efficient Vision Transformerscited by 133
  30. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 95
  31. 2020Bayesian Bits: Unifying Quantization and Pruningcited by 94
  32. 2020Generative Low-bitwidth Data Free Quantizationcited by 88
  33. 2021Diversifying Sample Generation for Accurate Data-Free Quantizationcited by 63
  34. 2021Annealing Knowledge Distillationcited by 61
  35. 2021Learning Efficient Vision Transformers via Fine-Grained Manifold Distillationcited by 60
  36. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  37. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 56
  38. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 55

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 15; 2025: 13; 2026: 8.

42 papers.

Sort:

By area:

  1. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  2. 2024A Survey on Transformer Compressioncited by 46
  3. 2023A Survey of Techniques for Optimizing Transformer Inferencecited by 35
  4. 2025Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategiescited by 24
  5. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Modelscited by 23
  6. 2024Towards Accurate Post-Training Quantization of Vision Transformers via Error Reductioncited by 23
  7. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  8. 2023Communication-Efficient Split Learning via Adaptive Feature-Wise Compressioncited by 18
  9. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  10. 2023Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precisioncited by 17
  11. 2024CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTscited by 17
  12. 2024A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trendscited by 16
30 more
  1. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  2. 2024RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationcited by 10
  3. 2024MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similaritycited by 9
  4. 2024Dataset Size Recovery from LoRA Weightscited by 7
  5. 2025OuroMamba: A Data-Free Quantization Framework for Vision Mambacited by 5
  6. 2024Privacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcarecited by 4
  7. 2025Janus: Collaborative Vision Transformer Under Dynamic Network Environmentcited by 4
  8. 2025Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applicationscited by 4
  9. 2026K-Sort Eval: Efficient Preference Evaluation for Visual Generation via Corrected VLM-as-a-Judgecited by 4
  10. 2024Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformerscited by 3
  11. 2025SAQ-SAM: Semantically-Aligned Quantization for Segment Anything Modelcited by 3
  12. 2024QuantTune: Optimizing Model Quantization with Adaptive Outlier-Driven Fine Tuningcited by 2
  13. 2025Stabilizing Quantization-Aware Training by Implicit-Regularization on Hessian Matrixcited by 2
  14. 2025DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuningcited by 2
  15. 2025Communication Efficient Split Learning of ViTs with Attention-based Double Compressioncited by 2
  16. 2026Efficient-SAM2: Accelerating SAM2 with Object-Aware Visual Encoding and Memory Retrievalcited by 2
  17. 2022CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformerscited by 1
  18. 2024In-Distribution Consistency Regularization Improves the Generalization of Quantization-Aware Trainingcited by 1
  19. 2024Mixed-precision Supernet Training from Vision Foundation Models using Low Rank Adaptercited by 1
  20. 2025UAV-Assisted Real-Time Disaster Detection Using Optimized Transformer Modelcited by 1
  21. 2025Efficiently Training A Flat Neural Network Before It has been Quantizatedcited by 1
  22. 2025IPTQ-ViT: Post-Training Quantization of Non-linear Functions for Integer-only Vision Transformerscited by 1
  23. 2026Joint Post-Training Quantization of Vision Transformers with Learned Prompt-Guided Data Generationcited by 1
  24. 2026SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuningcited by 1
  25. 2026Selective Coupling of Decoupled Informative Regions: Masked Attention Alignment for Data-Free Quantization of Vision Transformerscited by 1
  26. 2025Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compressioncited by 0
  27. 2025D4C: Data-Free Quantization for Contrastive Language-Image Pre-training Modelscited by 0
  28. 2026Big2Small: A Unifying Neural Network Framework for Model Compressioncited by 0
  29. 2026Enabling Fully Integer-Only Inference for Lightweight Detection Transformerscited by 0
  30. 2026IDeaL: Data-Free Multi-Teacher Distillation via Improved Dead Leavescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
  2. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization
  3. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
  4. 2024ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers
  5. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
  6. 2026ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers
  7. 2024Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer
  8. 2023Towards Accurate Post-Training Quantization for Vision Transformer