mlsanityData to 7 Oct 2026

MTMamba++: Enhancing Multi-Task Dense Scene Understanding via Mamba-Based Decoders

arXiv 2408.15101 ↗First posted 27 Aug 2024cs.CV cs.AIcited by 10

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

Multi-task dense scene understanding, which trains a model for multiple dense prediction tasks, has a wide range of application scenarios. Capturing long-range dependency and enhancing cross-task interactions are crucial to multi-task dense prediction. In this paper, we propose MTMamba++, a novel architecture for multi-task scene understanding featuring with a Mamba-based decoder. It contains two types of core blocks: self-task Mamba (STM) block and cross-task Mamba (CTM) block. STM handles long-range dependency by leveraging state-space models, while CTM explicitly models task interactions to facilitate information exchange across tasks. We design two types of CTM block, namely F-CTM and S-CTM, to enhance cross-task interaction from feature and semantic perspectives, respectively. Extensive experiments on NYUDv2, PASCAL-Context, and Cityscapes datasets demonstrate the superior performance of MTMamba++ over CNN-based, Transformer-based, and diffusion-based methods while maintaining high computational efficiency. The code is available at https://github.com/EnVision-Research/MTMamba.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 44 of its 59 references that are in the map. First posted on arXiv 27 Aug 2024.

Builds on

44 of 59 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders2 Jul
  2. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model17 Jan

Other references · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  8. 2024VMamba: Visual State Space Modelcited by 955
  9. 2017A Survey on Multi-Task Learningcited by 838
  10. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  11. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  12. 2016Cross-stitch Networks for Multi-task Learningcited by 466
30 more
  1. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  2. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 412
  3. 2019Attention Augmented Convolutional Networkscited by 363
  4. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
  5. 2022Long Range Language Modeling via Gated State Spacescited by 265
  6. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 230
  7. 2018PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsingcited by 224
  8. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  9. 2020MTI-Net: Multi-Scale Task Interaction Networks for Multi-Task Learningcited by 157
  10. 2019Attentive Single-Tasking of Multiple Taskscited by 148
  11. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 141
  12. 2021CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attentioncited by 139
  13. 2024Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spacescited by 132
  14. 2021Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learningcited by 100
  15. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  16. 2022InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understandingcited by 89
  17. 2021P2T: Pyramid Pooling Transformer for Scene Understandingcited by 84
  18. 2024Graph Mamba: Towards Learning on Graphs with State Space Modelscited by 84
  19. 2024DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesiscited by 72
  20. 2021Exploring Relational Context for Multi-Task Dense Predictioncited by 71
  21. 2024MambaByte: Token-free Selective State Space Modelcited by 58
  22. 2024Multi-Task Dense Prediction via Mixture of Low-Rank Expertscited by 58
  23. 2024Is Mamba Capable of In-Context Learning?cited by 45
  24. 2021Multi-Objective Meta Learningcited by 39
  25. 2022GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learningcited by 35
  26. 2022Multi-Task Learning with Multi-Query Transformer for Dense Predictioncited by 34
  27. 2023Dual-Balancing for Multi-Task Learningcited by 27
  28. 2023InvPT++: Inverted Pyramid Multi-Task Transformer for Visual Scene Understandingcited by 22
  29. 2023TSP-Transformer: Task-Specific Prompts Boosted Transformer for Holistic Scene Understandingcited by 12
  30. 2024A First-Order Multi-Gradient Algorithm for Multi-Objective Bi-Level Optimizationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

  1. 2025Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyondcited by 15
  2. 2025MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generationcited by 8
  3. 2025M2H: Multi-Task Learning with Efficient Window-Based Cross-Task Attention for Monocular Spatial Perceptioncited by 3
  4. 2026UV-M3TL: A Unified and Versatile Multimodal Multi-Task Learning Framework for Assistive Driving Perceptioncited by 2
  5. 2026M2H-MX: Multi-Task Semantic and Geometric Perception for Real-Time Monocular 3D Scene Graph Constructioncited by 1
  6. 2026PDD: Manifold-Prior Diverse Distillation for Medical Anomaly Detectioncited by 0
  7. 2026B3-Net: Controlled Posterior Bridge Learning for Multi-Task Dense Predictioncited by 0
  8. 2026Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mappingcited by 0
  9. 2026DPNeXt: A Lightweight Multi-Scale Feature Fusion Framework for Efficient ViT-Based Multi-Task Dense Predictioncited by 0
  10. 2026Unified Video Dense Prediction from Disjoint Datacited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Mamba Decoder with Bidirectional Interaction in Multi-Task Dense Prediction
  2. 2025Parameter Aware Mamba Model for Multi-task Dense Prediction
  3. 2024The Hidden Attention of Mamba Models
  4. 2024MobileMamba: Lightweight Multi-Receptive Visual Mamba Network
  5. 2025A2Mamba: Attention-augmented State Space Models for Visual Recognition
  6. 2024A Novel State Space Model with Local Enhancement and State Sharing for Image Fusion
  7. 2025MaTVLM: Hybrid Mamba-Transformer for Efficient Vision-Language Modeling
  8. 2025VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representation