mlsanityData to 7 Oct 2026

ReMamber: Referring Image Segmentation with Mamba Twister

arXiv 2403.17839 ↗First posted 26 Mar 2024cs.CV cs.AIcited by 38

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

Referring Image Segmentation~(RIS) leveraging transformers has achieved great success on the interpretation of complex visual-language tasks. However, the quadratic computation cost makes it resource-consuming in capturing long-range visual-language dependencies. Fortunately, Mamba addresses this with efficient linear complexity in processing. However, directly applying Mamba to multi-modal interactions presents challenges, primarily due to inadequate channel interactions for the effective fusion of multi-modal data. In this paper, we propose ReMamber, a novel RIS architecture that integrates the power of Mamba with a multi-modal Mamba Twister block. The Mamba Twister explicitly models image-text interaction, and fuses textual and visual features through its unique channel and spatial twisting mechanism. We achieve competitive results on three challenging benchmarks with a simple and efficient architecture. Moreover, we conduct thorough analyses of ReMamber and discuss other fusion designs using Mamba. These provide valuable perspectives for future research. The code has been released at: https://github.com/yyh-rain-song/ReMamber.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 49 of its 60 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

49 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Pan-Mamba: Effective pan-sharpening with State Space Model19 Feb
  2. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model17 Jan

Other references · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  7. 2024VMamba: Visual State Space Modelcited by 955
  8. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  9. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  10. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  11. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  12. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
35 more
  1. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 427
  2. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  3. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 412
  4. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  5. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  6. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  7. 2024VM-UNet: Vision Mamba UNet for Medical Image Segmentationcited by 286
  8. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
  9. 2022Long Range Language Modeling via Gated State Spacescited by 265
  10. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  11. 2016Segmentation from Natural Language Expressionscited by 257
  12. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  13. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 250
  14. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  15. 2024Swin-UMamba: Mamba-based UNet with ImageNet-based pretrainingcited by 179
  16. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  17. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  18. 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
  19. 2017Recurrent Multimodal Interaction for Referring Image Segmentationcited by 125
  20. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  21. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  22. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  23. 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
  24. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  25. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  26. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  27. 2018Dynamic Multimodal Instance Segmentation guided by natural language queriescited by 83
  28. 2023Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentationcited by 61
  29. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  30. 2022Efficient Movie Scene Detection using State-Space Transformerscited by 51
  31. 2019Referring Expression Object Segmentation with Caption-Aware Consistencycited by 46
  32. 2024U-shaped Vision Mamba for Single Image Dehazingcited by 40
  33. 2022Open-vocabulary Semantic Segmentation with Frozen Vision-Language Modelscited by 39
  34. 2023AttrSeg: Open-Vocabulary Semantic Segmentation via Attribute Decomposition-Aggregationcited by 29
  35. 2022Position-Aware Contrastive Alignment for Referring Image Segmentationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 17; 2026: 8.

38 papers.

Sort:

By area:

  1. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  2. 2024Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Modelcited by 62
  3. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  4. 2024Vision Mamba: A Comprehensive Survey and Taxonomycited by 53
  5. 2024A Survey on Visual Mambacited by 50
  6. 2024Visual Mamba: A Survey and New Outlookscited by 41
  7. 2024VSSD: Vision Mamba with Non-Causal State Space Dualitycited by 29
  8. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  9. 2025Multimodal Referring Segmentation: A Surveycited by 22
  10. 2024Hybrid Mamba for Few-Shot Segmentationcited by 21
  11. 2025AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentationcited by 15
  12. 2024MambaCSR: Dual-Interleaved Scanning for Compressed Image Super-Resolution With SSMscited by 14
26 more
  1. 2025DeRIS: Decoupling Perception and Cognition for Enhanced Referring Image Segmentation through Loopback Synergycited by 13
  2. 2025A Large-Scale Referring Remote Sensing Image Segmentation Dataset and Benchmarkcited by 12
  3. 2025WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentationcited by 9
  4. 2025Latent Expression Generation for Referring Image Segmentation and Groundingcited by 6
  5. 2024FACTS: A Factored State-Space Framework For World Modellingcited by 4
  6. 2025MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognitioncited by 4
  7. 2025SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentationcited by 3
  8. 2025VL4Gaze: Unleashing Vision-Language Models for Gaze Followingcited by 3
  9. 2026GenMask: Adapting DiT for Segmentation via Direct Mask Generationcited by 3
  10. 2024MaskRIS: Semantic Distortion-aware Data Augmentation for Referring Image Segmentationcited by 2
  11. 2025EDMB: Edge Detector with Mambacited by 2
  12. 2025Omni-Referring Image Segmentationcited by 2
  13. 2026SSP-SAM: SAM with Semantic-Spatial Prompt for Referring Expression Segmentationcited by 2
  14. 2024Venturing into Uncharted Waters: The Navigation Compass from Transformer to Mambacited by 1
  15. 2024MamKPD: A Simple Mamba Baseline for Real-Time 2D Keypoint Detectioncited by 1
  16. 2025A Comprehensive Survey on Magnetic Resonance Image Reconstructioncited by 1
  17. 2025RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentationcited by 1
  18. 2025Layover or Direct Flight: Rethinking Audio-Guided Image Segmentationcited by 1
  19. 2026TALENT: Target-aware Efficient Tuning for Referring Image Segmentationcited by 1
  20. 2025LMDepth: Lightweight Mamba-based Monocular Depth Estimation for Real-World Deploymentcited by 0
  21. 2025TFANet: Three-Stage Image-Text Feature Alignment Network for Robust Referring Image Segmentationcited by 0
  22. 2026Spatio-Semantic Expert Routing Architecture with Mixture-of-Experts for Referring Image Segmentationcited by 0
  23. 2026Boosting Text-Driven Video Segmentation via Geometry-Aware Distillationcited by 0
  24. 2026TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generationcited by 0
  25. 2026DeCo: Efficient Decouple-to-Couple Learning for Multi-Task Visual Groundingcited by 0
  26. 20260.5%>100%: Bidirectional Reciprocal Learning for Referring Image Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SF-Mamba: Rethinking State Space Model for Vision
  2. 2025Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentation
  3. 2024A Novel State Space Model with Local Enhancement and State Sharing for Image Fusion
  4. 2024ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2
  5. 2025MambaSeg: Harnessing Mamba for Accurate and Efficient Image-Event Semantic Segmentation
  6. 2024LoG-VMamba: Local-Global Vision Mamba for Medical Image Segmentation
  7. 2025MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks
  8. 2024LocalMamba: Visual State Space Model with Windowed Selective Scan