mlsanityData to 7 Oct 2026

FTerViT: Fully Ternary Vision Transformer

arXiv 2605.21171 ↗First posted 20 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Ternary Vision Transformers offer substantial model compression, however state-of-the-art methods only ternarize the encoder layers, leaving patch embeddings, LayerNorm parameters, and classifier heads in full precision. In compact models targeting resource-constrained processors, such as microcontrollers, these remaining full-precision components determine the total memory footprint, severely limiting deployment efficiency and on-device feasibility. In this work, we introduce a fully ternarized Vision Transformer in which all weight matrices and normalization parameters are ternarized (FTerViT). To this end, we introduce two novel operators : TernaryBitConv2d with per-channel scaling for patch embedding and TernaryLayerNorm. FTerViT is trained using knowledge distillation, followed by a lightweight quantization-aware recovery phase. Our ternary W2A8 DeiT-III-S at 384×384 resolution achieves 82.43% ImageNet-1K top-1 at 6.09,MB (~15× compression, -2.42,pp vs. FP32), outperforming prior ternary ViTs methods up to 8 pp. Finally, we demonstrate the first implementation of ternary vision transformers on a dual cores XTensa LX7 microcontroller inside the ESP32-S3 system-on-chip. By deploying FTerViT-Small (based on DeiT-III-Small at 224×224 resolution, 5.81,MB), we achieve 79.64% ImageNet-1K top-1 accuracy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 51 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

46 of 51 references.

References · 46

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  5. 2020Quantifying Attention Flow in Transformerscited by 759
  6. 2019Learned Step Size Quantizationcited by 604
  7. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  8. 2019Importance Estimation for Neural Network Pruningcited by 531
  9. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  10. 2022DeiT III: Revenge of the ViTcited by 417
  11. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  12. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
34 more
  1. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  2. 2021Post-Training Quantization for Vision Transformercited by 274
  3. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  4. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 232
  5. 2016Ternary Weight Networkscited by 195
  6. 2020ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functionscited by 183
  7. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  8. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  9. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  10. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
  11. 2024Logit Standardization in Knowledge Distillationcited by 101
  12. 2022I-ViT: Integer-only Quantization for Efficient Vision Transformer Inferencecited by 93
  13. 2024LLM Pruning and Distillation in Practice: The Minitron Approachcited by 82
  14. 2023Oscillation-free Quantization for Low-bit Vision Transformerscited by 55
  15. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  16. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 20
  17. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  18. 2024Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokenscited by 18
  19. 2023BinaryViT: Pushing Binary Vision Transformers Towards Convolutional Modelscited by 17
  20. 2023Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precisioncited by 17
  21. 2023TinyTracker: Ultra-Fast and Ultra-Low-Power Edge Vision In-Sensor for Gaze Estimationcited by 16
  22. 2023Bi-ViT: Pushing the Limit of Vision Transformer Quantizationcited by 13
  23. 2022Understanding and Improving Knowledge Distillation for Quantization-Aware Training of Large Transformer Encoderscited by 11
  24. 2022TerViT: An Efficient Ternary Vision Transformercited by 10
  25. 2023Poster: Self-Supervised Quantization-Aware Knowledge Distillationcited by 10
  26. 2023MCUFormer: Deploying Vision Transformers on Microcontrollers with Limited Memorycited by 9
  27. 2023The Role of Masking for Efficient Supervised Knowledge Distillation of Vision Transformerscited by 8
  28. 2024LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagationcited by 8
  29. 2023BinaryViT: Towards Efficient and Accurate Binary Vision Transformerscited by 7
  30. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Eracited by 7
  31. 2025Can LLMs Revolutionize the Design of Explainable and Efficient TinyML Models?cited by 5
  32. 2025TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledgecited by 4
  33. 2025LL-ViT: Edge Deployable Vision Transformers with Look Up Table Neuronscited by 3
  34. 2023TinyFormer: Efficient Transformer Design and Deployment on Tiny Devicescited by 2

Built on by

Not cited yet.