mlsanityData to 7 Oct 2026

Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces

arXiv 2403.19925 ↗First posted 29 Mar 2024cs.LG cs.AIcited by 23

On the map: State-space sequence models (S4, Mamba) → Video, motion and generative Mamba models

also in 1 more
Abstract (arXiv)

Decision Transformer, a promising approach that applies Transformer architectures to reinforcement learning, relies on causal self-attention to model sequences of states, actions, and rewards. While this method has shown competitive results, this paper investigates the integration of the Mamba framework, known for its advanced capabilities in efficient and effective sequence modeling, into the Decision Transformer architecture, focusing on the potential performance enhancements in sequential decision-making tasks. Our study systematically evaluates this integration by conducting a series of experiments across various decision-making environments, comparing the modified Decision Transformer, Decision Mamba, with its traditional counterpart. This work contributes to the advancement of sequential decision-making models, suggesting that the architecture and training methodology of neural networks can significantly impact their performance in complex tasks, and highlighting the potential of Mamba as a valuable tool for improving the efficacy of Transformer-based models in reinforcement learning scenarios.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Video, motion and generative Mamba models, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 14 of its 19 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

14 of 19 references.

References · 14

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2016OpenAI Gymcited by 1,684
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  8. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  9. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  10. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  11. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  12. 2021Mastering Atari Games with Limited Datacited by 235
2 more
  1. 2023Decision S4: Efficient Sequence-Based RL via State Spaces Layerscited by 26
  2. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Makingcited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 7; 2026: 4.

23 papers.

Sort:

By area:

  1. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  2. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  3. 2024Were RNNs All We Needed?cited by 36
  4. 2024TOP-ERL: Transformer-based Off-Policy Episodic Reinforcement Learningcited by 18
  5. 2024A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Taskscited by 14
  6. 2024Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?cited by 7
  7. 2025ELMUR: External Layer Memory with Update/Rewrite for Long-Horizon RL Problemscited by 6
  8. 2024Drama: Mamba-Enabled Model-Based Reinforcement Learning Is Sample and Parameter Efficientcited by 4
  9. 2024Multi-Agent Reinforcement Learning with Selective State-Space Modelscited by 2
  10. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  11. 2025RLBenchNet: The Right Network for the Right Reinforcement Learning Taskcited by 2
  12. 2025Memba: Membrane-driven Parameter-Efficient Fine-Tuning for Mambacited by 2
11 more
  1. 2024Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learningcited by 1
  2. 2024Demystifying the Token Dynamics of Deep Selective State Space Modelscited by 1
  3. 2026DADP: Domain Adaptive Diffusion Policycited by 1
  4. 2024Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixingcited by 0
  5. 2024Decision Transformer vs. Decision Mamba: Analysing the Complexity of Sequential Decision Making in Atari Gamescited by 0
  6. 2025Do We Need Transformers to Play FPS Video Games?cited by 0
  7. 2025Success in Humanoid Reinforcement Learning under Partial Observationcited by 0
  8. 2025Unified token representations for sequential decision modelscited by 0
  9. 2026Decision MetaMamba: Enhancing Selective SSM in Offline RL with Heterogeneous Sequence Mixingcited by 0
  10. 2026Belief-State RWKV for Reinforcement Learning under Partial Observabilitycited by 0
  11. 2026QHyer: Q-conditioned Hybrid Attention-mamba Transformer for Offline Goal-conditioned RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling
  2. 2024Decision Mamba Architectures
  3. 2024Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL
  4. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  5. 2022Hierarchical Decision Transformer
  6. 2025Scaling Algorithm Distillation for Continuous Control with Mamba
  7. 2024In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
  8. 2024Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learning