mlsanityData to 14 Oct 2026

CAMMARL: Conformal Action Modeling in Multi Agent Reinforcement Learning

arXiv 2306.11128 ↗First posted 19 Jun 2023cs.LG cs.MAcited by 3

Not on the map. Why?

Abstract (arXiv)

Before taking actions in an environment with more than one intelligent agent, an autonomous agent may benefit from reasoning about the other agents and utilizing a notion of a guarantee or confidence about the behavior of the system. In this article, we propose a novel multi-agent reinforcement learning (MARL) algorithm CAMMARL, which involves modeling the actions of other agents in different situations in the form of confident sets, i.e., sets containing their true actions with a high probability. We then use these estimates to inform an agent's decision-making. For estimating such sets, we use the concept of conformal predictions, by means of which, we not only obtain an estimate of the most probable outcome but get to quantify the operable uncertainty as well. For instance, we can predict a set that provably covers the true predictions with high probabilities (e.g., 95%). Through several experiments in two fully cooperative multi-agent tasks, we show that CAMMARL elevates the capabilities of an autonomous agent in MARL by modeling conformal prediction sets over the behavior of other agents in the environment and utilizing such estimates to enhance its policy learning.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 19 Jun 2023.

Builds on

32 of 49 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2020Mastering Atari with Discrete World Modelscited by 1,007
  5. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  6. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  7. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  8. 2016Learning Multiagent Communication with Backpropagationcited by 366
  9. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 366
  10. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  11. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
  12. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 260
20 more
  1. 2018Machine Theory of Mindcited by 226
  2. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 209
  3. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
  4. 2017Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problemscited by 131
  5. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 104
  6. 2016Opponent Modeling in Deep Reinforcement Learningcited by 97
  7. 2020Learning Latent Representations to Influence Multi-Agent Interactioncited by 93
  8. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 71
  9. 2019PAC Confidence Sets for Deep Neural Networks via Calibrated Predictioncited by 60
  10. 2020Agent Modelling under Partial Observability for Deep Reinforcement Learningcited by 57
  11. 2018Learning Policy Representations in Multiagent Systemscited by 48
  12. 2015A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systemscited by 44
  13. 2018Cautious Deep Learningcited by 35
  14. 2021Deep Interactive Bayesian Reinforcement Learning via Meta-Learningcited by 29
  15. 2021Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rolloutscited by 24
  16. 2016Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networkscited by 21
  17. 2018Relational Forward Models for Multi-Agent Learningcited by 21
  18. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  19. 2021HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messagingcited by 13
  20. 2022Learning Robust Dynamics through Variational Sparse Gatingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Unifying Agent Interaction and World Information for Multi-agent Coordinationcited by 2
  2. 2026Do LLM-derived graph priors improve multi-agent coordination?cited by 0
  3. 2026Statistical Convergence of Transformer Encoder-Accelerated Robust Reinforcement Learningcited by 0