mlsanityData to 14 Oct 2026

MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders

arXiv 2407.02228 ↗First posted 2 Jul 2024cs.CV cs.AIcited by 22

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

Multi-task dense scene understanding, which learns a model for multiple dense prediction tasks, has a wide range of application scenarios. Modeling long-range dependency and enhancing cross-task interactions are crucial to multi-task dense prediction. In this paper, we propose MTMamba, a novel Mamba-based architecture for multi-task scene understanding. It contains two types of core blocks: self-task Mamba (STM) block and cross-task Mamba (CTM) block. STM handles long-range dependency by leveraging Mamba, while CTM explicitly models task interactions to facilitate information exchange across tasks. Experiments on NYUDv2 and PASCAL-Context datasets demonstrate the superior performance of MTMamba over Transformer-based and CNN-based methods. Notably, on the PASCAL-Context dataset, MTMamba achieves improvements of +2.08, +5.01, and +4.90 over the previous best methods in the tasks of semantic segmentation, human parsing, and object boundary detection, respectively. The code is available at https://github.com/EnVision-Research/MTMamba.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 39 of its 53 references that are in the map. First posted on arXiv 2 Jul 2024.

Builds on

39 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model17 Jan

Other references · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2020Gradient Surgery for Multi-Task Learningcited by 1,256
  8. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
  9. 2024VMamba: Visual State Space Modelcited by 969
  10. 2017A Survey on Multi-Task Learningcited by 840
  11. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  12. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 608
26 more
  1. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  2. 2016Cross-stitch Networks for Multi-task Learningcited by 467
  3. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  4. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 419
  5. 2019Attention Augmented Convolutional Networkscited by 363
  6. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
  7. 2022Long Range Language Modeling via Gated State Spacescited by 265
  8. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 232
  9. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  10. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  11. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  12. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 142
  13. 2024Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spacescited by 134
  14. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 101
  15. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  16. 2022InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understandingcited by 89
  17. 2024Graph Mamba: Towards Learning on Graphs with State Space Modelscited by 86
  18. 2021Exploring Relational Context for Multi-Task Dense Predictioncited by 71
  19. 2024MambaByte: Token-free Selective State Space Modelcited by 59
  20. 2021Multi-task Learning with Attention for End-to-end Autonomous Drivingcited by 49
  21. 2024Is Mamba Capable of In-Context Learning?cited by 45
  22. 2021Multi-Objective Meta Learningcited by 39
  23. 2022GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learningcited by 36
  24. 2022Multi-Task Learning with Multi-Query Transformer for Dense Predictioncited by 34
  25. 2023Dual-Balancing for Multi-Task Learningcited by 27
  26. 2024A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimizationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 7; 2026: 10.

22 papers.

Sort:

By area:

  1. 2024A Survey of Mambacited by 39
  2. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 18
  3. 2024AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignmentcited by 17
  4. 2025FlowRAM: Grounding Flow Matching Policy with Region-Aware Mamba Framework for Robotic Manipulationcited by 13
  5. 2024MTMamba++: Enhancing Multi-Task Dense Scene Understanding via Mamba-Based Decoderscited by 10
  6. 2025MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generationcited by 8
  7. 2024Swiss Army Knife: Synergizing Biases in Knowledge from Vision Foundation Models for Multi-Task Learningcited by 7
  8. 2024RAWMamba: Unified sRGB-to-RAW De-rendering With State Space Modelcited by 3
  9. 2025UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensationcited by 3
  10. 2025M2H: Multi-Task Learning with Efficient Window-Based Cross-Task Attention for Monocular Spatial Perceptioncited by 3
  11. 2025InterMamba: Efficient Human-Human Interaction Generation with Adaptive Spatio-Temporal Mambacited by 2
  12. 2026UV-M3TL: A Unified and Versatile Multimodal Multi-Task Learning Framework for Assistive Driving Perceptioncited by 2
10 more
  1. 2025Enhancing Mamba Decoder with Bidirectional Interaction in Multi-Task Dense Predictioncited by 1
  2. 2026SMTrack: State-Aware Mamba for Efficient Temporal Modeling in Visual Trackingcited by 1
  3. 2026M2H-MX: Multi-Task Semantic and Geometric Perception for Real-Time Monocular 3D Scene Graph Constructioncited by 1
  4. 2026DIFF-MF: A Difference-Driven Channel-Spatial State Space Model for Multi-Modal Image Fusioncited by 0
  5. 2026Task Prototype-Based Knowledge Retrieval for Multi-Task Learning from Partially Annotated Datacited by 0
  6. 2026B3-Net: Controlled Posterior Bridge Learning for Multi-Task Dense Predictioncited by 0
  7. 2026Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mappingcited by 0
  8. 2026DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Predictioncited by 0
  9. 2026Unified Video Dense Prediction from Disjoint Datacited by 0
  10. 2026LiFR v2: Completion-Augmented Event Propagation for High-Rate Dense Predictioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Parameter Aware Mamba Model for Multi-task Dense Prediction
  2. 2024MambaOut: Do We Really Need Mamba for Vision?
  3. 2025MaTVLM: Hybrid Mamba-Transformer for Efficient Vision-Language Modeling
  4. 2024Sigma: Siamese Mamba Network for Multi-Modal Semantic Segmentation
  5. 2024The Hidden Attention of Mamba Models
  6. 2025Mamba as a Bridge: Where Vision Foundation Models Meet Vision Language Models for Domain-Generalized Semantic Segmentation
  7. 2025MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks
  8. 2024A Novel State Space Model with Local Enhancement and State Sharing for Image Fusion