mlsanityData to 14 Oct 2026

RLBenchNet: The Right Network for the Right Reinforcement Learning Task

arXiv 2505.15040 ↗First posted 21 May 2025cs.LGcited by 2

On the map: State-space sequence models (S4, Mamba) → Video, motion and generative Mamba models

also in 1 more
Abstract (arXiv)

Reinforcement learning (RL) has seen significant advancements through the application of various neural network architectures. In this study, we systematically investigate the performance of several neural networks in RL tasks, including Long Short-Term Memory (LSTM), Multi-Layer Perceptron (MLP), Mamba/Mamba-2, Transformer-XL, Gated Transformer-XL, and Gated Recurrent Unit (GRU). Through comprehensive evaluation across continuous control, discrete decision-making, and memory-based environments, we identify architecture-specific strengths and limitations. Our results reveal that: (1) MLPs excel in fully observable continuous control tasks, providing an optimal balance of performance and efficiency; (2) recurrent architectures like LSTM and GRU offer robust performance in partially observable environments with moderate memory requirements; (3) Mamba models achieve a 4.5x higher throughput compared to LSTM and a 3.9x increase over GRU, all while maintaining comparable performance; and (4) only Transformer-XL, Gated Transformer-XL, and Mamba-2 successfully solve the most challenging memory-intensive tasks, with Mamba-2 requiring 8x less memory than Transformer-XL. These findings provide insights for researchers and practitioners, enabling more informed architecture selection based on specific task characteristics and computational constraints. Code is available at: https://github.com/SafeRL-Lab/RLBenchNet

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Video, motion and generative Mamba models, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 14 of its 24 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

14 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Decision Mamba: A Multi-Grained State Space Model with Self-Evolution Regularization for Offline RL8 Jun
  2. 2024Decision Mamba: Reinforcement Learning via Sequence Modeling with Selective State Spaces29 Mar

Other references · 12

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  6. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  7. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  8. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 162
  9. 2023When Do Transformers Shine in RL? Decoupling Memory from Credit Assignmentcited by 63
  10. 2023Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agentscited by 16
  11. 2024Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Ratecited by 10
  12. 2024Safe and Balanced: A Framework for Constrained Multi-Objective Reinforcement Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Driftcited by 1
  2. 2026Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Investigating Action Encodings in Recurrent Neural Networks in Reinforcement Learning
  2. 2024Real-Time Recurrent Learning using Trace Units in Reinforcement Learning
  3. 2026Does Learning Recurrent Networks Matter in Partially Observable RL?
  4. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
  5. 2025A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report
  6. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models
  7. 2026Why Linear Recurrent Memory Works in Partially Observable Reinforcement Learning
  8. 2026Beyond Isolation: Unlocking Reinforcement Learning Component Synergy for Sample-Efficient Continuous Control