mlsanityData to 14 Oct 2026

RPFQ-ViT: Rotated Phase-Frame Quantization for Extremely Low-Bit Weights in Vision Transformers

arXiv 2610.04457 ↗First posted 3 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) achieve strong performance on image recognition and mobile vision applications, but their high-dimensional linear projections and attention computations still impose substantial storage and inference costs. Extremely low-bit quantization is a promising solution, yet ViTs often suffer severe accuracy degradation because conventional real-valued scalar codebooks are poorly matched to the directional geometry of Transformer projections. We present RPFQ-ViT, a Rotated Phase-Frame Quantization method that quantizes paired channels in two-dimensional phase planes, enabling low-bit codes to better preserve projection directions while recovering magnitude with lightweight scaling. RPFQ-ViT serves as a drop-in QAT replacement for nn.Linear and does not modify the standard real-valued attention, normalization, or activation computation graph. On ImageNet-1K, RPFQ-ViT-B/16 reaches 79.33% Top-1 / 94.48% Top-5 under W2/A4, Swin-T reaches 79.30% Top-1 / 94.79% Top-5 under W2/A8, and DeiT-S reaches 77.41% Top-1 / 93.11% Top-5 under W2/A8. Ablations, phase-geometry analysis, and direction-preservation metrics show that channel pairing, learnable rotation, phase-anchor learning, and residual phase refinement each improve quantization quality. We further deploy RPFQ-ViT image-classification models on native iOS and Android runtime stacks; with 2-bit packed weights, model size shrinks by roughly 5.4-7.1× relative to FP32 and end-to-end on-device latency drops by 1.4-1.6×. All ImageNet results trained in our codebase use a matched 300-epoch recipe and are reported as mean accuracies over three independent runs. These results show that RPFQ-ViT provides a favorable trade-off among accuracy, compression, and practical mobile deployment for extremely low-bit ViTs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 3 Oct 2026.

Builds on

21 of 24 references.

References · 21

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  6. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  8. 2019Learned Step Size Quantizationcited by 628
  9. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  10. 2017Deep Complex Networkscited by 217
  11. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 189
  12. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
9 more
  1. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  2. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  3. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 108
  4. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 57
  5. 2024Instance-Aware Group Quantization for Vision Transformerscited by 25
  6. 2025APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformerscited by 19
  7. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximationcited by 19
  8. 2024FrameQuant: Flexible Low-Bit Quantization for Transformerscited by 18
  9. 2025GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformerscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
  2. 2024MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity
  3. 2025LampQ: Towards Accurate Layer-wise Mixed Precision Quantization for Vision Transformers
  4. 2024P2-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
  5. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer
  6. 2023TSPTQ-ViT: Two-scaled post-training quantization for vision transformer
  7. 2025IPTQ-ViT: Post-Training Quantization of Non-linear Functions for Integer-only Vision Transformers
  8. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization