# Patch Similarity Aware Data-Free Quantization for Vision Transformers

arXiv: https://arxiv.org/abs/2203.02250 · HTML: https://mlsanity.com/paper/2203.02250/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 43 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 30 of 37)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [DeepViT: Towards Deeper Vision Transformer](https://mlsanity.com/paper/2103.11886/) (2021)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Rethinking and Improving Relative Position Encoding for Vision Transformer](https://mlsanity.com/paper/2107.14222/) (2021)
- [Low-bit Quantization of Neural Networks for Efficient Inference](https://mlsanity.com/paper/1902.06822/) (2019)
- [PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization](https://mlsanity.com/paper/2111.12293/) (2021)
- [Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks](https://mlsanity.com/paper/1909.13144/) (2019)
- [Patch Slimming for Efficient Vision Transformers](https://mlsanity.com/paper/2106.02852/) (2021)

## Built on by (most cited first, 30 of 56)

- [RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers](https://mlsanity.com/paper/2212.08254/) (2022)
- [I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference](https://mlsanity.com/paper/2207.01405/) (2022)
- [Towards Accurate Post-training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.18723/) (2023)
- [Q-VLM: Post-training Quantization for Large Vision-Language Models](https://mlsanity.com/paper/2410.08119/) (2024)
- [A Survey on Transformer Compression](https://mlsanity.com/paper/2402.05964/) (2024)
- [PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers](https://mlsanity.com/paper/2209.05687/) (2022)
- [A Survey of Techniques for Optimizing Transformer Inference](https://mlsanity.com/paper/2307.07982/) (2023)
- [EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Models](https://mlsanity.com/paper/2401.04585/) (2024)
- [PowerQuant: Automorphism Search for Non-Uniform Quantization](https://mlsanity.com/paper/2301.09858/) (2023)
- [Boost Vision Transformer with GPU-Friendly Sparsity and Quantization](https://mlsanity.com/paper/2305.10727/) (2023)
- [I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization](https://mlsanity.com/paper/2311.10126/) (2023)
- [DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers](https://mlsanity.com/paper/2408.03291/) (2024)
- [CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs](https://mlsanity.com/paper/2407.05266/) (2024)
- [K-Sort Arena: Efficient and Reliable Benchmarking for Generative Models via K-wise Human Preferences](https://mlsanity.com/paper/2408.14468/) (2024)
- [Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey](https://mlsanity.com/paper/2405.00314/) (2024)
- [CacheQuant: Comprehensively Accelerated Diffusion Models](https://mlsanity.com/paper/2503.01323/) (2025)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [Transformers in Single Object Tracking: An Experimental Survey](https://mlsanity.com/paper/2302.11867/) (2023)
- [QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources](https://mlsanity.com/paper/2310.07147/) (2023)
- [RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterization](https://mlsanity.com/paper/2402.05628/) (2024)
- [MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity](https://mlsanity.com/paper/2407.20021/) (2024)
- [Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity](https://mlsanity.com/paper/2501.06357/) (2025)
- [GenQ: Quantization in Low Data Regimes with Generative Synthetic Data](https://mlsanity.com/paper/2312.05272/) (2023)
- [LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation](https://mlsanity.com/paper/2401.11243/) (2024)
- [Zero-shot Quantization: A Comprehensive Survey](https://mlsanity.com/paper/2505.09188/) (2025)
- [TSPTQ-ViT: Two-scaled post-training quantization for vision transformer](https://mlsanity.com/paper/2305.12901/) (2023)
- [BinaryViT: Towards Efficient and Accurate Binary Vision Transformers](https://mlsanity.com/paper/2305.14730/) (2023)
- [PQD: Post-training Quantization for Efficient Diffusion Models](https://mlsanity.com/paper/2501.00124/) (2024)
- [OuroMamba: A Data-Free Quantization Framework for Vision Mamba](https://mlsanity.com/paper/2503.10959/) (2025)
- [GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samples](https://mlsanity.com/paper/2305.07931/) (2023)

## Abstract (arXiv)

Vision transformers have recently gained great success on various computer vision tasks; nevertheless, their high model complexity makes it challenging to deploy on resource-constrained devices. Quantization is an effective approach to reduce model complexity, and data-free quantization, which can address data privacy and security concerns during model deployment, has received widespread interest. Unfortunately, all existing methods, such as BN regularization, were designed for convolutional neural networks and cannot be applied to vision transformers with significantly different model architectures. In this paper, we propose PSAQ-ViT, a Patch Similarity Aware data-free Quantization framework for Vision Transformers, to enable the generation of "realistic" samples based on the vision transformer's unique properties for calibrating the quantization parameters. Specifically, we analyze the self-attention module's properties and reveal a general difference (patch similarity) in its processing of Gaussian noise and real images. The above insights guide us to design a relative value metric to optimize the Gaussian noise to approximate the real images, which are then utilized to calibrate the quantization parameters. Extensive experiments and ablation studies are conducted on various benchmarks to validate the effectiveness of PSAQ-ViT, which can even outperform the real-data-driven methods. Code is available at: https://github.com/zkkli/PSAQ-ViT.