mlsanityData to 14 Oct 2026

Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning

arXiv 2207.04978 ↗First posted 11 Jul 2022cs.CV cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

Multi-scale Vision Transformer (ViT) has emerged as a powerful backbone for computer vision tasks, while the self-attention computation in Transformer scales quadratically w.r.t. the input patch number. Thus, existing solutions commonly employ down-sampling operations (e.g., average pooling) over keys/values to dramatically reduce the computational cost. In this work, we argue that such over-aggressive down-sampling design is not invertible and inevitably causes information dropping especially for high-frequency components in objects (e.g., texture details). Motivated by the wavelet theory, we construct a new Wavelet Vision Transformer (Wave-ViT) that formulates the invertible down-sampling with wavelet transforms and self-attention learning in a unified way. This proposal enables self-attention learning with lossless down-sampling over keys/values, facilitating the pursuing of a better efficiency-vs-accuracy trade-off. Furthermore, inverse wavelet transforms are leveraged to strengthen self-attention outputs by aggregating local contexts with enlarged receptive field. We validate the superiority of Wave-ViT through extensive experiments over multiple vision tasks (e.g., image recognition, object detection and instance segmentation). Its performances surpass state-of-the-art ViT backbones with comparable FLOPs. Source code is available at urlhttps://github.com/YehLi/ImageNetModel.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Builds on 67 of its 78 references that are in the map. First posted on arXiv 11 Jul 2022.

Builds on

67 of 78 references.

References · 67

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2017Mask R-CNNcited by 7,277
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  10. 2017Squeeze-and-Excitation Networkscited by 4,789
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  12. 2014Going Deeper with Convolutionscited by 4,241
38 more
  1. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  3. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  4. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  5. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  6. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  7. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  8. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  9. 2017Random Erasing Data Augmentationcited by 1,239
  10. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  11. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  12. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  13. 2021Multiscale Vision Transformerscited by 897
  14. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  15. 2021Going deeper with Image Transformerscited by 764
  16. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  17. 2019Panoptic Feature Pyramid Networkscited by 645
  18. 2021Transformer in Transformercited by 633
  19. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 582
  20. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  21. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  22. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 540
  23. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  24. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  25. 2019Res2Net: A New Multi-scale Backbone Architecturecited by 470
  26. 2020ResNeSt: Split-Attention Networkscited by 445
  27. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 409
  28. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  29. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  30. 2019Attention Augmented Convolutional Networkscited by 363
  31. 2020Exploring Self-attention for Image Recognitioncited by 313
  32. 2021DeepViT: Towards Deeper Vision Transformercited by 289
  33. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  34. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  35. 2019Local Relation Networks for Image Recognitioncited by 210
  36. 2020X-Linear Attention Networks for Image Captioningcited by 198
  37. 2018Multi-level Wavelet-CNN for Image Restorationcited by 184
  38. 2021BEVT: BERT Pretraining of Video Transformerscited by 169

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 18; 2024: 22; 2025: 15; 2026: 18.

78 papers.

Sort:

By area:

  1. 2023BiFormer: Vision Transformer with Bi-Level Routing Attentioncited by 151
  2. 2022Wavelet Diffusion Models are fast and scalable Image Generatorscited by 117
  3. 2024SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time seriescited by 82
  4. 2023RMT: Retentive Networks Meet Vision Transformerscited by 68
  5. 2024VideoStudio: Generating Consistent-Content and Multi-Scene Videoscited by 64
  6. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 64
  7. 2023SpectFormer: Frequency and Attention is what you need in a Vision Transformercited by 48
  8. 2023Control3D: Towards Controllable Text-to-3D Generationcited by 46
  9. 2024Visual Mamba: A Survey and New Outlookscited by 41
  10. 2023ControlStyle: Text-Driven Stylized Image Generation Using Diffusion Priorscited by 40
  11. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  12. 2022Hierarchical Local-Global Transformer for Temporal Sentence Groundingcited by 38
38 more
  1. 2024TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Modelscited by 31
  2. 2023Lightweight Vision Transformer with Bidirectional Interactioncited by 25
  3. 2023Ultra-High Resolution Segmentation with Ultra-Rich Context: A Novel Benchmarkcited by 22
  4. 2022Jigsaw-ViT: Learning Jigsaw Puzzles in Vision Transformercited by 21
  5. 2024LoFormer: Local Frequency Transformer for Image Deblurringcited by 20
  6. 2025Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imagingcited by 20
  7. 2022Deep Convolutional Pooling Transformer for Deepfake Detectioncited by 19
  8. 2023Scattering Vision Transformer: Spectral Mixing Matterscited by 17
  9. 2023Guided Patch-Grouping Wavelet Transformer with Spatial Congruence for Ultra-High Resolution Segmentationcited by 16
  10. 2024Spectral Image Tokenizercited by 15
  11. 2025WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detectioncited by 14
  12. 2024LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognitioncited by 13
  13. 2024HIRI-ViT: Scaling Vision Transformer with High Resolution Inputscited by 12
  14. 2025Vision Transformers with Self-Distilled Registerscited by 12
  15. 2022WavEnhancer: Unifying Wavelet and Transformer for Image Enhancementcited by 11
  16. 2024DiMSUM: Diffusion Mamba -- A Scalable and Unified Spatial-Frequency Method for Image Generationcited by 11
  17. 2024Wavelet-Based Image Tokenizer for Vision Transformerscited by 10
  18. 2023Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformerscited by 9
  19. 2023UWFormer: Underwater Image Enhancement via a Semi-Supervised Multi-Scale Transformercited by 9
  20. 2025Dual-domain Modulation Network for Lightweight Image Super-Resolutioncited by 8
  21. 2023A Convolutional Vision Transformer for Semantic Segmentation of Side-Scan Sonar Datacited by 7
  22. 2023Spatial-Frequency U-Net for Denoising Diffusion Probabilistic Modelscited by 7
  23. 2024WIA-LD2ND: Wavelet-based Image Alignment for Self-supervised Low-Dose CT Denoisingcited by 7
  24. 2023Efficiency 360: Efficient Vision Transformerscited by 6
  25. 2024Heracles: A Hybrid SSM-Transformer Model for High-Resolution Image and Time-Series Analysiscited by 5
  26. 2025ClusIR: Towards Cluster-Guided All-in-One Image Restorationcited by 5
  27. 2023ProtoHPE: Prototype-guided High-frequency Patch Enhancement for Visible-Infrared Person Re-identificationcited by 4
  28. 2023Unlocking Fine-Grained Details with Wavelet-based High-Frequency Enhancement in Transformerscited by 3
  29. 2024FViT: A Focal Vision Transformer with Gabor Filtercited by 3
  30. 2024WaveDH: Wavelet Sub-bands Guided ConvNet for Efficient Image Dehazingcited by 3
  31. 2025Wavelet-Driven Masked Image Modeling: A Path to Efficient Visual Representationcited by 3
  32. 2026HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNetcited by 3
  33. 2024Spectral U-Net: Enhancing Medical Image Segmentation via Spectral Decompositioncited by 2
  34. 2024Window-based Channel Attention for Wavelet-enhanced Learned Image Compressioncited by 2
  35. 2025Beyond Fixed Patches: Enhancing GPTs for Financial Prediction with Adaptive Segmentation and Learnable Waveletscited by 2
  36. 2026MFEN:Multi-Frequency Expert Network for Visible-Infrared Person Re-IDcited by 2
  37. 2026WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensationcited by 2
  38. 2024MxT: Mamba x Transformer for Image Inpaintingcited by 1

50 most cited of 78 All 78 (JSON)