mlsanityData to 14 Oct 2026

SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series

arXiv 2403.15360 ↗First posted 22 Mar 2024cs.CV cs.LG cs.SYcited by 82

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

Transformers have widely adopted attention networks for sequence mixing and MLPs for channel mixing, playing a pivotal role in achieving breakthroughs across domains. However, recent literature highlights issues with attention networks, including low inductive bias and quadratic complexity concerning input sequence length. State Space Models (SSMs) like S4 and others (Hippo, Global Convolutions, liquid S4, LRU, Mega, and Mamba), have emerged to address the above issues to help handle longer sequence lengths. Mamba, while being the state-of-the-art SSM, has a stability issue when scaled to large networks for computer vision datasets. We propose SiMBA, a new architecture that introduces Einstein FFT (EinFFT) for channel modeling by specific eigenvalue computations and uses the Mamba block for sequence modeling. Extensive performance studies across image and time-series benchmarks demonstrate that SiMBA outperforms existing SSMs, bridging the performance gap with state-of-the-art transformers. Notably, SiMBA establishes itself as the new state-of-the-art SSM on ImageNet and transfer learning benchmarks such as Stanford Car and Flower as well as task learning benchmarks as well as seven time series benchmark datasets. The project page is available on this website ~urlhttps://github.com/badripatro/Simba.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 57 of its 73 references that are in the map. First posted on arXiv 22 Mar 2024.

Builds on

57 of 73 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2017Mask R-CNNcited by 7,277
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  9. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  10. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  11. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  12. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
38 more
  1. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  2. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  3. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  4. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  5. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  6. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  7. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  8. 2024VMamba: Visual State Space Modelcited by 969
  9. 2020Designing Network Design Spacescited by 960
  10. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  11. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  12. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  13. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  14. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  15. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 566
  16. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  17. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
  18. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  19. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  20. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 409
  21. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 393
  22. 2021FNet: Mixing Tokens with Fourier Transformscited by 347
  23. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 338
  24. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 332
  25. 2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
  26. 2022Long Range Language Modeling via Gated State Spacescited by 265
  27. 2021Global Filter Networks for Image Classificationcited by 260
  28. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
  29. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 188
  30. 2022Mega: Moving Average Equipped Gated Attentioncited by 179
  31. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 172
  32. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 165
  33. 2020Liquid Time-constant Networkscited by 161
  34. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 148
  35. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 127
  36. 2022Inception Transformercited by 99
  37. 2022Fast Vision Transformers with HiLo Attentioncited by 78
  38. 2022Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learningcited by 78

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 38; 2025: 21; 2026: 23.

82 papers.

Sort:

By area:

  1. 2024MambaVision: A Hybrid Mamba-Transformer Vision Backbonecited by 224
  2. 2024MambaOut: Do We Really Need Mamba for Vision?cited by 143
  3. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 64
  4. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  5. 2024Vision Mamba: A Comprehensive Survey and Taxonomycited by 54
  6. 2024A Survey on Visual Mambacited by 50
  7. 2024Samba: Semantic Segmentation of Remotely Sensed Images with State Space Modelcited by 37
  8. 2025Directing Mamba to Complex Textures: An Efficient Texture-Aware State Space Model for Image Restorationcited by 28
  9. 2024QMambaBSR: Burst Image Super-Resolution with Query State Space Modelcited by 27
  10. 2024Dimba: Transformer-Mamba Diffusion Modelscited by 26
  11. 2024Mambular: A Sequential Model for Tabular Deep Learningcited by 21
  12. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videoscited by 18
38 more
  1. 2024Mamba in Vision: A Comprehensive Survey of Techniques and Applicationscited by 18
  2. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  3. 2024GroupMamba: Efficient Group-Based Visual State Space Modelcited by 15
  4. 2024ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attentioncited by 13
  5. 2024S3Mamba: Arbitrary-Scale Super-Resolution via Scaleable State Space Modelcited by 12
  6. 2024DiMSUM: Diffusion Mamba -- A Scalable and Unified Spatial-Frequency Method for Image Generationcited by 11
  7. 2025BiCrossMamba-ST: Speech Deepfake Detection with Bidirectional Mamba Spectro-Temporal Cross-Attentioncited by 10
  8. 2024Time-SSM: Simplifying and Unifying State Space Models for Time Series Forecastingcited by 9
  9. 2024QMamba: On First Exploration of Vision Mamba for Image Quality Assessmentcited by 9
  10. 2024Chimera: Effectively Modeling Multivariate Time Series with 2-Dimensional State Space Modelscited by 8
  11. 2024CMamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecastingcited by 8
  12. 2025MamBEV: Enabling State Space Models to Learn Birds-Eye-View Representationscited by 8
  13. 2025AutoHFormer: Efficient Hierarchical Autoregressive Transformer for Time Series Predictioncited by 8
  14. 2025Dynamic Vision Mambacited by 6
  15. 2025Vision Mamba in Remote Sensing: A Comprehensive Survey of Techniques, Applications and Outlookcited by 6
  16. 2024Heracles: A Hybrid SSM-Transformer Model for High-Resolution Image and Time-Series Analysiscited by 5
  17. 2024Towards a theory of learning dynamics in deep state space modelscited by 5
  18. 2024Faster Vision Mamba is Rebuilt in Minutes via Merged Token Re-trainingcited by 5
  19. 2025SparseSSM: Efficient Selective Structured State Space Models Can Be Pruned in One-Shotcited by 5
  20. 2024PowerMamba: A Deep State Space Model and Comprehensive Benchmark for Time Series Prediction in Electric Power Systemscited by 4
  21. 2025FLDmamba: Integrating Fourier and Laplace Transform Decomposition with Mamba for Enhanced Time Series Predictioncited by 4
  22. 2024CLIP-Mamba: CLIP Pretrained Mamba Models with OOD and Hessian Evaluationcited by 3
  23. 2024UmambaTSF: A U-shaped Multi-Scale Long-Term Time Series Forecasting Method Using Mambacited by 3
  24. 2025Global-Aware Monocular Semantic Scene Completion with State Space Modelscited by 3
  25. 2026HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNetcited by 3
  26. 2024MLLA-UNet: Mamba-like Linear Attention in an Efficient U-Shape Model for Medical Image Segmentationcited by 2
  27. 2025Let SSMs be ConvNets: State-space Modeling with Optimal Tensor Contractionscited by 2
  28. 2025Technologies on Effectiveness and Efficiency: A Survey of State Spaces Modelscited by 2
  29. 2025An Exploration of Mamba for Speech Self-Supervised Modelscited by 2
  30. 2025Exploring State-Space-Model based Language Model in Music Generationcited by 2
  31. 2026Moving Beyond Functional Connectivity: Time-Series Modeling for fMRI-Based Brain Disorder Classificationcited by 2
  32. 2024HRSAM: Efficient Interactive Segmentation in High-Resolution Imagescited by 1
  33. 2024EDCSSM: Edge Detection with Convolutional State Space Modelcited by 1
  34. 2024PASS: Path-selective State Space Model for Event-based Recognitioncited by 1
  35. 2024GlobalMamba: Global Image Serialization for Vision Mambacited by 1
  36. 2024Variable Aperture Bokeh Rendering via Customized Focal Plane Guidancecited by 1
  37. 2024MTS-UNMixers: Multivariate Time Series Forecasting via Channel-Time Dual Unmixingcited by 1
  38. 2024SkelMamba: A State Space Model for Efficient Skeleton Action Recognition of Neurological Disorderscited by 1

50 most cited of 82 All 82 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Mamba: A Survey and New Outlooks
  2. 2025Fast Vision Mamba: Pooling Spatial Dimensions for Accelerated Processing
  3. 2024Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data
  4. 2024Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model
  5. 2025Visual Attention Exploration in Vision-Based Mamba Models
  6. 2025Achilles' Heel of Mamba: Essential difficulties of the Mamba architecture demonstrated by synthetic data
  7. 2024LocalMamba: Visual State Space Model with Windowed Selective Scan
  8. 2024Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis