mlsanityData to 14 Oct 2026

Sub-token ViT Embedding via Stochastic Resonance Transformers

arXiv 2310.03967 ↗First posted 6 Oct 2023cs.CV cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Vision Transformer (ViT) architectures represent images as collections of high-dimensional vectorized tokens, each corresponding to a rectangular non-overlapping patch. This representation trades spatial granularity for embedding dimensionality, and results in semantically rich but spatially coarsely quantized feature maps. In order to retrieve spatial details beneficial to fine-grained inference tasks we propose a training-free method inspired by "stochastic resonance". Specifically, we perform sub-token spatial transformations to the input data, and aggregate the resulting ViT features after applying the inverse transformation. The resulting "Stochastic Resonance Transformer" (SRT) retains the rich semantic information of the original representation, but grounds it on a finer-scale spatial domain, partly mitigating the coarse effect of spatial tokenization. SRT is applicable across any layer of any ViT architecture, consistently boosting performance on several tasks including segmentation, classification, depth estimation, and others by up to 14.9% without the need for any fine-tuning.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 33 of its 53 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

33 of 53 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2023Segment Anythingcited by 8,234
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  11. 2014Going Deeper with Convolutionscited by 4,241
  12. 2021Vision Transformers for Dense Predictioncited by 1,803
21 more
  1. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  2. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  3. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  4. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  5. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  6. 2021Deep ViT Features as Dense Visual Descriptorscited by 330
  7. 2022How Do Vision Transformers Work?cited by 307
  8. 2018nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentationcited by 291
  9. 2018Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networkscited by 280
  10. 2017PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examplescited by 244
  11. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  12. 2018Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarkingcited by 191
  13. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  14. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 159
  15. 2022TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cutcited by 113
  16. 2018Deflecting Adversarial Attacks with Pixel Deflectioncited by 71
  17. 2019Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attackscited by 54
  18. 2023Emergence of Segmentation with Minimalistic White-Box Transformerscited by 24
  19. 2017Image Classification of Melanoma, Nevus and Seborrheic Keratosis by Deep Neural Network Ensemblecited by 23
  20. 2021Efficient Self-Ensemble for Semantic Segmentationcited by 16
  21. 2014Domain-Size Pooling in Local Descriptors: DSP-SIFTcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 5; 2026: 2.

9 papers.

  1. 2024RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptionscited by 21
  2. 2023AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimationcited by 15
  3. 2025ProtoDepth: Unsupervised Continual Depth Completion with Prototypescited by 11
  4. 2025Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokenscited by 9
  5. 2025Radar-Guided Polynomial Fitting for Metric Depth Estimationcited by 5
  6. 2025ETA: Energy-based Test-time Adaptation for Depth Completioncited by 4
  7. 2025Test-Time Defense Against Adversarial Attacks via Stochastic Resonance of Latent Ensemblescited by 2
  8. 2026VisTa3D: A Dataset and Benchmark for Thin Object Reconstruction from Vision, Tactile, and 3D Point Cloudscited by 1
  9. 2026From Perspective to Fisheye Depth Estimation and Open-Vocabulary Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ViT-5: Vision Transformers for The Mid-2020s
  2. 2021Discrete Representations Strengthen Vision Transformer Robustness
  3. 2023MSViT: Dynamic Mixed-Scale Tokenization for Vision Transformers
  4. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization
  5. 2025Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI
  6. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
  7. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformer
  8. 2024Vision Transformers with Natural Language Semantics