mlsanityData to 14 Oct 2026

Scalable, Decentralized Multi-Agent Reinforcement Learning Methods Inspired by Stigmergy and Ant Colonies

arXiv 2105.03546 ↗First posted 8 May 2021cs.MA cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Bolstering multi-agent learning algorithms to tackle complex coordination and control tasks has been a long-standing challenge of on-going research. Numerous methods have been proposed to help reduce the effects of non-stationarity and unscalability. In this work, we investigate a novel approach to decentralized multi-agent learning and planning that attempts to address these two challenges. In particular, this method is inspired by the cohesion, coordination, and behavior of ant colonies. As a result, these algorithms are designed to be naturally scalable to systems with numerous agents. While no optimality is guaranteed, the method is intended to work well in practice and scale better in efficacy with the number of agents present than others. The approach combines single-agent RL and an ant-colony-inspired decentralized, stigmergic algorithm for multi-agent path planning and environment modification. Specifically, we apply this algorithm in a setting where agents must navigate to a goal location, learning to push rectangular boxes into holes to yield new traversable pathways. It is shown that while the approach yields promising success in this particular environment, it may not be as easily generalized to others. The algorithm designed is notably scalable to numerous agents but is limited in its performance due to its relatively simplistic, rule-based approach. Furthermore, the composability of RL-trained policies is called into question, where, while policies are successful in their training environments, applying trained policies to a larger-scale, multi-agent framework results in unpredictable behavior.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 8 May 2021.

Builds on

21 of 36 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  9. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  10. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  11. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  12. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
9 more
  1. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  2. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  3. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  4. 2016Opponent Modeling in Deep Reinforcement Learningcited by 97
  5. 2018Modeling Others using Oneself in Multi-Agent Reinforcement Learningcited by 66
  6. 2019Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learningcited by 33
  7. 2019Deep Q-Learning for Nash Equilibria: Nash-DQNcited by 14
  8. 2019Stigmergic Independent Reinforcement Learning for Multi-Agent Collaborationcited by 8
  9. 2019Learning Hierarchical Control for Robust In-Hand Manipulationcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Deep Reinforcement Learning for Multi-Agent Coordinationcited by 0