mlsanityData to 14 Oct 2026

Sable: a Performant, Efficient and Scalable Sequence Model for MARL

arXiv 2410.01706 ↗First posted 2 Oct 2024cs.LG cs.AI cs.MAcited by 10

Not on the map. Why?

Abstract (arXiv)

As multi-agent reinforcement learning (MARL) progresses towards solving larger and more complex problems, it becomes increasingly important that algorithms exhibit the key properties of (1) strong performance, (2) memory efficiency, and (3) scalability. In this work, we introduce Sable, a performant, memory-efficient, and scalable sequence modeling approach to MARL. Sable works by adapting the retention mechanism in Retentive Networks (Sun et al., 2023) to achieve computationally efficient processing of multi-agent observations with long context memory for temporal reasoning. Through extensive evaluations across six diverse environments, we demonstrate how Sable is able to significantly outperform existing state-of-the-art methods in a large number of diverse tasks (34 out of 45 tested). Furthermore, Sable maintains performance as we scale the number of agents, handling environments with more than a thousand agents while exhibiting a linear increase in memory usage. Finally, we conduct ablation studies to isolate the source of Sable's performance gains and confirm its efficient computational memory usage.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 53 of its 81 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

53 of 81 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2016Layer Normalizationcited by 5,669
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  7. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  9. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  10. 2020GLU Variants Improve Transformercited by 1,804
  11. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  12. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
38 more
  1. 2018Group Normalizationcited by 1,524
  2. 2019Root Mean Square Layer Normalizationcited by 1,480
  3. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  6. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  7. 2019The StarCraft Multi-Agent Challengecited by 597
  8. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  9. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  10. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  11. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  12. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  13. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  14. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 270
  15. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 212
  16. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  17. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 184
  18. 2020FACMAC: Factored Multi-Agent Centralised Policy Gradientscited by 178
  19. 2022V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Drivingcited by 157
  20. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  21. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 116
  22. 2023Structured State Space Models for In-Context Reinforcement Learningcited by 111
  23. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 104
  24. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 98
  25. 2021Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharingcited by 87
  26. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  27. 2023Heterogeneous-Agent Reinforcement Learningcited by 69
  28. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 58
  29. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  30. 2020Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learningcited by 48
  31. 2021UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformerscited by 39
  32. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 33
  33. 2021Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Taskscited by 32
  34. 2024Open RL Benchmark: Comprehensive Tracked Experiments for Reinforcement Learningcited by 17
  35. 2021Mava: a research library for distributed multi-agent reinforcement learning in JAXcited by 16
  36. 2022Heterogeneous-Agent Mirror Learning: A Continuum of Solutions to Cooperative MARLcited by 12
  37. 2024Improving Token-Based World Models with Parallel Observation Predictioncited by 11
  38. 2023TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problemscited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2025Multi-Agent Guided Policy Optimizationcited by 3
  2. 2025Breaking the Performance Ceiling in Reinforcement Learning requires Inference Strategiescited by 2
  3. 2026Randomness is sometimes necessary for coordinationcited by 1
  4. 2025Oryx: a Scalable Sequence Model for Many-Agent Coordination in Offline MARLcited by 0
  5. 2025A Survey of Retentive Networkcited by 0
  6. 2026Characterizing MARL for Energy Control: A Multi-KPI Benchmark on the CityLearn Environmentcited by 0
  7. 2026Hierarchical Lead Critic based Multi-Agent Reinforcement Learningcited by 0
  8. 2026Rethinking Ratio-Based Trust Regions for Policy Optimization in Multi-Agent Reinforcement Learningcited by 0
  9. 2026Self-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisationcited by 0
  10. 2026Out-of-Distribution Generalisation with Sequence Models in Offline Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When Do Multi-Agent Systems Outperform? Analysing the Learning Efficiency of Agentic Systems
  2. 2025Remembering the Markov Property in Cooperative MARL
  3. 2026SEARL: Joint Optimization of Policy and Tool Graph Memory for Self-Evolving Agents
  4. 2026Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
  5. 2024Unraveling the Complexity of Memory in RL Agents: an Approach for Classification and Evaluation
  6. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
  7. 2026Toward Efficient Agents: Memory, Tool learning, and Planning
  8. 2026MARTI-MARS2: Scaling Multi-Agent Self-Search via Reinforcement Learning for Code Generation