mlsanityData to 7 Oct 2026

Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures

arXiv 2403.02308 ↗First posted 4 Mar 2024cs.CVcited by 97

Not on the map. Why?

Abstract (arXiv)

Transformers have revolutionized computer vision and natural language processing, but their high computational complexity limits their application in high-resolution image processing and long-context analysis. This paper introduces Vision-RWKV (VRWKV), a model adapted from the RWKV model used in the NLP field with necessary modifications for vision tasks. Similar to the Vision Transformer (ViT), our model is designed to efficiently handle sparse inputs and demonstrate robust global processing capabilities, while also scaling up effectively, accommodating both large-scale parameters and extensive datasets. Its distinctive advantage lies in its reduced spatial aggregation complexity, which renders it exceptionally adept at processing high-resolution images seamlessly, eliminating the necessity for windowing operations. Our evaluations demonstrate that VRWKV surpasses ViT's performance in image classification and has significantly faster speeds and lower memory usage processing high-resolution inputs. In dense prediction tasks, it outperforms window-based models, maintaining comparable speeds. These results highlight VRWKV's potential as a more efficient alternative for visual perception tasks. Code is released at https://github.com/OpenGVLab/Vision-RWKV.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 54 of its 68 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

54 of 68 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  10. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  11. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  12. 2017Mask R-CNNcited by 7,259
38 more
  1. 2016Densely Connected Convolutional Networkscited by 6,691
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2016Layer Normalizationcited by 5,579
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2014Going Deeper with Convolutionscited by 4,231
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  13. 2017Deformable Convolutional Networkscited by 1,481
  14. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  15. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  16. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  17. 2024VMamba: Visual State Space Modelcited by 955
  18. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  19. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  20. 2021Going deeper with Image Transformerscited by 759
  21. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  22. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  23. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  24. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  25. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  26. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  27. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  28. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 479
  29. 2019Selective Kernel Networkscited by 401
  30. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  31. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  32. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  33. 2018Online normalizer calculation for softmaxcited by 154
  34. 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
  35. 2023Hierarchically Gated Recurrent Neural Network for Sequence Modelingcited by 100
  36. 2023RMT: Retentive Networks Meet Vision Transformerscited by 68
  37. 2024Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applicationscited by 49
  38. 2023Toeplitz Neural Network for Sequence Modelingcited by 43

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 30; 2025: 42; 2026: 25.

97 papers.

Sort:

By area:

  1. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  2. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  3. 2024Vision-LSTM: xLSTM as Generic Vision Backbonecited by 51
  4. 2024Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Modelscited by 41
  5. 2024LION: Linear Group RNN for 3D Object Detection in Point Cloudscited by 38
  6. 2024Restore-RWKV: Efficient and Effective Medical Image Restoration with RWKVcited by 36
  7. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  8. 2024RWKV-CLIP: A Robust Vision-Language Representation Learnercited by 23
  9. 2025Linear Attention Modeling for Learned Image Compressioncited by 22
  10. 2024PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learningcited by 21
  11. 2024Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Modelcited by 18
  12. 2024On Efficient Variants of Segment Anything Model: A Surveycited by 16
38 more
  1. 2024MobileMamba: Lightweight Multi-Receptive Visual Mamba Networkcited by 16
  2. 2024ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attentioncited by 13
  3. 2024Explaining Modern Gated-Linear RNNs via a Unified Implicit Attention Formulationcited by 11
  4. 2024Event Stream based Human Action Recognition: A High-Definition Benchmark Dataset and Algorithmscited by 11
  5. 2024A Survey on Multimodal Wearable Sensor-based Human Action Recognitioncited by 10
  6. 2024Audio Mamba: Pretrained Audio State Space Model For Audio Taggingcited by 9
  7. 2024You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNetcited by 9
  8. 2024OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexitycited by 9
  9. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  10. 2025AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understandingcited by 9
  11. 2025RWKV-UNet: Improving UNet with Long-Range Cooperation for Effective Medical Image Segmentationcited by 7
  12. 2025RGB-Event based Pedestrian Attribute Recognition: A Benchmark Dataset and An Asymmetric RWKV Fusion Frameworkcited by 7
  13. 2025TransMamba: Fast Universal Architecture Adaption from Transformers to Mambacited by 6
  14. 2024EMOv2: Pushing 5M Vision Model Frontiercited by 5
  15. 2024A Survey of RWKVcited by 5
  16. 2025LOCORE: Image Re-ranking with Long-Context Sequence Modelingcited by 5
  17. 2025ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Modelscited by 5
  18. 2025Overflow Prevention Enhances Long-Context Recurrent LLMscited by 5
  19. 2025C3-OWD: A Curriculum Cross-modal Contrastive Learning Framework for Open-World Detectioncited by 5
  20. 2024Video RWKV:Video Action Recognition Based RWKVcited by 4
  21. 2025Parallel Sequence Modeling via Generalized Spatial Propagation Networkcited by 4
  22. 2025Thicker and Quicker: A Jumbo Token for Fast Plain Vision Transformerscited by 4
  23. 2025RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantizationcited by 4
  24. 2024Learning 1D Causal Visual Representation with De-focus Attention Networkscited by 3
  25. 2024Enhancing Environmental Monitoring through Multispectral Imaging: The WasteMS Dataset for Semantic Segmentation of Lakeside Wastecited by 3
  26. 2024Exploring Real&Synthetic Dataset and Linear Attention in Image Restorationcited by 3
  27. 2025S2AFormer: Strip Self-Attention for Efficient Vision Transformercited by 3
  28. 2025FEAT: Full-Dimensional Efficient Attention Transformer for Medical Video Generationcited by 3
  29. 2025SAGA: Selective Adaptive Gating for Efficient and Expressive Linear Attentioncited by 3
  30. 2025UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNscited by 3
  31. 2026EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecastingcited by 3
  32. 2026Innovative Tooth Segmentation Using Hierarchical Features and Bidirectional Sequence Modelingcited by 3
  33. 2026HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNetcited by 3
  34. 2026LaplacianFormer:Rethinking Linear Attention with Laplacian Kernelcited by 3
  35. 2024Experimentation in Content Moderation using RWKVcited by 2
  36. 2024SCASeg: Strip Cross-Attention for Efficient Semantic Segmentationcited by 2
  37. 2024StyleRWKV: High-Quality and High-Efficiency Style Transfer with RWKV-like Architecturecited by 2
  38. 2025Multi-View Learning with Context-Guided Receptance for Image Denoisingcited by 2

50 most cited of 97 All 97 (JSON)