mlsanityData to 7 Oct 2026

DeMansia: Mamba Never Forgets Any Tokens

arXiv 2408.01986 ↗First posted 4 Aug 2024cs.CV cs.AIcited by 0

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

This paper examines the mathematical foundations of transformer architectures, highlighting their limitations particularly in handling long sequences. We explore prerequisite models such as Mamba, Vision Mamba (ViM), and LV-ViT that pave the way for our proposed architecture, DeMansia. DeMansia integrates state space models with token labeling techniques to enhance performance in image classification tasks, efficiently addressing the computational challenges posed by traditional transformers. The architecture, benchmark, and comparisons with contemporary models demonstrate DeMansia's effectiveness. The implementation of this paper is available on GitHub at https://github.com/catalpaaa/DeMansia

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 21 of its 26 references that are in the map. First posted on arXiv 4 Aug 2024.

Builds on

21 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model17 Jan

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  9. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  10. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  11. 2017Mixed Precision Trainingcited by 1,135
  12. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 763
8 more
  1. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  2. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 273
  3. 2022TinyViT: Fast Pretraining Distillation for Small Vision Transformerscited by 232
  4. 2021All Tokens Matter: Token Labeling for Training Better Vision Transformerscited by 144
  5. 2022On The Computational Complexity of Self-Attentioncited by 137
  6. 2022Formal Algorithms for Transformerscited by 70
  7. 2020Linear Attention Mechanism: An Efficient Attention for Semantic Segmentationcited by 21
  8. 2023Adam Accumulation to Reduce Memory Footprints of both Activations and Gradients for Large-scale DNN Trainingcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey on Visual Mamba
  2. 2024A Survey of Mamba
  3. 2025Training-free Token Reduction for Vision Mamba
  4. 2024Mamba in Vision: A Comprehensive Survey of Techniques and Applications
  5. 2025DYNAMAX: Dynamic computing for Transformers and Mamba based architectures
  6. 2024MambaVision: A Hybrid Mamba-Transformer Vision Backbone
  7. 2025A Survey on Mamba Architecture for Vision Applications
  8. 2026Attention to Mamba: A Recipe for Cross-Architecture Distillation