# CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs

arXiv: https://arxiv.org/abs/2407.05266 · HTML: https://mlsanity.com/paper/2407.05266/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 52 references that are in the map.
- First posted on arXiv 7 Jul 2024.

## Builds on (most cited first, 30 of 38)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [EfficientFormer: Vision Transformers at MobileNet Speed](https://mlsanity.com/paper/2206.01191/) (2022)
- [PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization](https://mlsanity.com/paper/2111.12293/) (2021)
- [FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer](https://mlsanity.com/paper/2111.13824/) (2021)
- [Decoupled Contrastive Learning](https://mlsanity.com/paper/2110.06848/) (2021)
- [RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers](https://mlsanity.com/paper/2212.08254/) (2022)
- [Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer](https://mlsanity.com/paper/2210.06707/) (2022)
- [I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference](https://mlsanity.com/paper/2207.01405/) (2022)
- [CompletionFormer: Depth Completion with Convolutions and Vision Transformers](https://mlsanity.com/paper/2304.13030/) (2023)
- [Patch Similarity Aware Data-Free Quantization for Vision Transformers](https://mlsanity.com/paper/2203.02250/) (2022)
- [Robust Contrastive Learning against Noisy Views](https://mlsanity.com/paper/2201.04309/) (2022)
- [IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization](https://mlsanity.com/paper/2111.09136/) (2021)
- [PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers](https://mlsanity.com/paper/2209.05687/) (2022)
- [Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples](https://mlsanity.com/paper/2111.02625/) (2021)
- [An Introduction to Image Synthesis with Generative Adversarial Nets](https://mlsanity.com/paper/1803.04469/) (2018)
- [EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization](https://mlsanity.com/paper/2307.10554/) (2023)
- [Probabilistic Binary Neural Networks](https://mlsanity.com/paper/1809.03368/) (2018)

## Built on by (most cited first, 17 of 17)

- [MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization](https://mlsanity.com/paper/2411.05282/) (2024)
- [Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies](https://mlsanity.com/paper/2503.02891/) (2025)
- [QT-DoG: Quantization-aware Training for Domain Generalization](https://mlsanity.com/paper/2410.06020/) (2024)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity](https://mlsanity.com/paper/2407.20021/) (2024)
- [Zero-shot Quantization: A Comprehensive Survey](https://mlsanity.com/paper/2505.09188/) (2025)
- [OuroMamba: A Data-Free Quantization Framework for Vision Mamba](https://mlsanity.com/paper/2503.10959/) (2025)
- [LVLM-Compress-Bench: Benchmarking the Broader Impact of Large Vision-Language Model Compression](https://mlsanity.com/paper/2503.04982/) (2025)
- [Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers](https://mlsanity.com/paper/2412.16553/) (2024)
- [AIRCHITECT v2: Learning the Hardware Accelerator Design Space through Unified Representations](https://mlsanity.com/paper/2501.09954/) (2025)
- [IPTQ-ViT: Post-Training Quantization of Non-linear Functions for Integer-only Vision Transformers](https://mlsanity.com/paper/2511.15369/) (2025)
- [Joint Post-Training Quantization of Vision Transformers with Learned Prompt-Guided Data Generation](https://mlsanity.com/paper/2602.18861/) (2026)
- [SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning](https://mlsanity.com/paper/2603.18423/) (2026)
- [Selective Coupling of Decoupled Informative Regions: Masked Attention Alignment for Data-Free Quantization of Vision Transformers](https://mlsanity.com/paper/2606.04373/) (2026)
- [QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention](https://mlsanity.com/paper/2604.25306/) (2026)
- [UniDFKD: A Unified Semantic Prior Framework for Architecture-Agnostic Data-Free Knowledge Distillation](https://mlsanity.com/paper/2608.09287/) (2026)
- [IDeaL: Data-Free Multi-Teacher Distillation via Improved Dead Leaves](https://mlsanity.com/paper/2608.24759/) (2026)

## Abstract (arXiv)

We present CLAMP-ViT, a data-free post-training quantization method for vision transformers (ViTs). We identify the limitations of recent techniques, notably their inability to leverage meaningful inter-patch relationships, leading to the generation of simplistic and semantically vague data, impacting quantization accuracy. CLAMP-ViT employs a two-stage approach, cyclically adapting between data generation and model quantization. Specifically, we incorporate a patch-level contrastive learning scheme to generate richer, semantically meaningful data. Furthermore, we leverage contrastive learning in layer-wise evolutionary search for fixed- and mixed-precision quantization to identify optimal quantization parameters while mitigating the effects of a non-smooth loss landscape. Extensive evaluations across various vision tasks demonstrate the superiority of CLAMP-ViT, with performance improvements of up to 3% in top-1 accuracy for classification, 0.6 mAP for object detection, and 1.5 mIoU for segmentation at similar or better compression ratio over existing alternatives. Code is available at https://github.com/georgia-tech-synergy-lab/CLAMP-ViT.git