mlsanityData to 7 Oct 2026

Attention Is not Everything: Efficient Alternatives for Vision

arXiv 2604.17439 ↗First posted 19 Apr 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recently computer vision has seen advancements mainly thanks to Transformer-based models. However many non-Transformer methods are still doing well being a direct competition of Transformer-based models. This review tries to present a comprehensive taxonomy of such methods and organize these methods into categories like convolution-based models, MLP-based models, state-space-based and more. These methods are looked at in terms of how efficient they are, how well they scale, how easy they are to understand and how robust they are. A total of 40 papers were chosen for this study. The goal is to give a view of non-Transformer methods and find out what challenges and opportunities exist for future computer vision research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 81 of its 106 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

81 of 106 references.

References · 81

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2023Segment Anythingcited by 8,158
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
38 more
  1. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2022A ConvNet for the 2020scited by 3,545
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  9. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  10. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  11. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  12. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  13. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  14. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  15. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  16. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  17. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  18. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  19. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  20. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  21. 2024VMamba: Visual State Space Modelcited by 955
  22. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  23. 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 720
  24. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  25. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  26. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  27. 2023LightGlue: Local Feature Matching at Light Speedcited by 583
  28. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  29. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 531
  30. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
  31. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 471
  32. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  33. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 464
  34. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 424
  35. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 412
  36. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  37. 2020MOT20: A benchmark for multi object tracking in crowded scenescited by 355
  38. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354

50 most cited of 81 All 81 (JSON)

Built on by

Not cited yet.