mlsanityData to 14 Oct 2026

Zero-Shot Quantization via Weight-Space Arithmetic

arXiv 2604.03420 ↗First posted 3 Apr 2026cs.CV cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

We show that robustness to post-training quantization (PTQ) is a transferable direction in weight space. We call this direction the quantization vector: extracted from a donor task by simple weight-space arithmetic, it can be used to patch a receiver model and improve post-PTQ Top-1 accuracy by up to 60 points in a 3-bit setting, without receiver-side quantization-aware training (QAT). Because the method requires no receiver training data, it provides a zero-shot, low-cost alternative to QAT for extremely low-bit deployment. Across four ViT scales and 22 image classification tasks, donor quantization vectors often yield substantial gains even when donor and receiver tasks differ markedly. We further prove rigorously that quantization vectors are well-defined and do not suffer from reparameterization symmetries, and provide a local geometric account of their effect. Together, these results suggest that quantization robustness can be partially isolated, reused, and transferred through simple weight-space algebra.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 60 of its 80 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

60 of 80 references.

References · 60

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  7. 2013Describing Textures in the Wildcited by 2,125
  8. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  9. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  11. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  12. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
38 more
  1. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  3. 2022Editing Models with Task Arithmeticcited by 1,120
  4. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  6. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  7. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  8. 2022MTEB: Massive Text Embedding Benchmarkcited by 640
  9. 2021A White Paper on Neural Network Quantizationcited by 472
  10. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  11. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  12. 2018Rotation Equivariant CNNs for Digital Pathologycited by 422
  13. 2018Deep Learning for Classical Japanese Literaturecited by 422
  14. 2022DeiT III: Revenge of the ViTcited by 419
  15. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  16. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  17. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 302
  18. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  19. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 252
  20. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
  21. 2024Evolutionary Optimization of Model Merging Recipescited by 208
  22. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  23. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  24. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 158
  25. 2024Task Singular Vectors: Reducing Task Interference in Model Mergingcited by 147
  26. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 139
  27. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  28. 2025MMTEB: Massive Multilingual Text Embedding Benchmarkcited by 125
  29. 2024Model merging with SVD to tie the Knotscited by 122
  30. 2019Loss Aware Post-training Quantizationcited by 97
  31. 2025No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspacescited by 96
  32. 2021I Wish I Would Have Loved This One, But I Didn't -- A Multilingual Dataset for Counterfactual Detection in Product Reviewscited by 45
  33. 2025Language Models are Injective and Hence Invertiblecited by 35
  34. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizercited by 29
  35. 2025TorchAO: PyTorch-Native Training-to-Serving Model Optimizationcited by 26
  36. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  37. 2023Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformerscited by 18
  38. 2025Training Dynamics Impact Post-Training Quantization Robustnesscited by 15

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The Undetected Damage of Quantization on Retrieval and How to Fix Itcited by 0
  2. 2026Weight Pair Encoding: Inducing a Smaller Grammar in Neural Network Weightscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin
  2. 2026Scale Sensitivity in Low-Bit Post-Training Quantization: Curvature of the Quantization Error Landscape
  3. 2024COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization
  4. 2026SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning
  5. 2023Oscillation-free Quantization for Low-bit Vision Transformers
  6. 2021A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
  7. 2025AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers
  8. 2026When Lower Reconstruction Loss Hurts: Distributionally Robust Refinement for Low-Bit LLM Quantization