mlsanityData to 14 Oct 2026

Melting Pot 2.0

arXiv 2211.13746 ↗First posted 24 Nov 2022cs.MA cs.AI cs.GTcited by 40

Not on the map. Why?

Abstract (arXiv)

Multi-agent artificial intelligence research promises a path to develop intelligent technologies that are more human-like and more human-compatible than those produced by "solipsistic" approaches, which do not consider interactions between agents. Melting Pot is a research tool developed to facilitate work on multi-agent artificial intelligence, and provides an evaluation protocol that measures generalization to novel social partners in a set of canonical test scenarios. Each scenario pairs a physical environment (a "substrate") with a reference set of co-players (a "background population"), to create a social situation with substantial interdependence between the individuals involved. For instance, some scenarios were inspired by institutional-economics-based accounts of natural resource management and public-good-provision dilemmas. Others were inspired by considerations from evolutionary biology, game theory, and artificial life. Melting Pot aims to cover a maximally diverse set of interdependencies and incentives. It includes the commonly-studied extreme cases of perfectly-competitive (zero-sum) motivations and perfectly-cooperative (shared-reward) motivations, but does not stop with them. As in real-life, a clear majority of scenarios in Melting Pot have mixed incentives. They are neither purely competitive nor purely cooperative and thus demand successful agents be able to navigate the resulting ambiguity. Here we describe Melting Pot 2.0, which revises and expands on Melting Pot. We also introduce support for scenarios with asymmetric roles, and explain how to integrate them into the evaluation protocol. This report also contains: (1) details of all substrates and scenarios; (2) a complete description of all baseline algorithms and results. Our intention is for it to serve as a reference for researchers using Melting Pot 2.0.

Key facts. Cited by 40 papers in the map (data through 14 Oct 2026). Builds on 46 of its 84 references that are in the map. First posted on arXiv 24 Nov 2022.

Builds on

46 of 84 references.

References · 46

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  4. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  5. 2016Concrete Problems in AI Safetycited by 1,642
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  8. 2021Ethical and social risks of harm from Language Modelscited by 966
  9. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  10. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  11. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
  12. 2018Quantifying Generalization in Reinforcement Learningcited by 352
34 more
  1. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  2. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  3. 2017Six Challenges for Neural Machine Translationcited by 335
  4. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
  5. 2016Cooperative Inverse Reinforcement Learningcited by 300
  6. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  7. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  8. 2017Learning with Opponent-Learning Awarenesscited by 222
  9. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 209
  10. 2020Open Problems in Cooperative AIcited by 168
  11. 2021Collaborating with Humans without Human Datacited by 142
  12. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
  13. 2018Generalization and Regularization in DQNcited by 118
  14. 2018Inequity aversion improves cooperation in intertemporal social dilemmascited by 106
  15. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
  16. 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
  17. 2018A Dissection of Overfitting and Generalization in Continuous Reinforcement Learningcited by 87
  18. 2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
  19. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
  20. 2016Research Priorities for Robust and Beneficial Artificial Intelligencecited by 74
  21. 2022Mapping global dynamics of benchmark creation and saturation in artificial intelligencecited by 71
  22. 2017A multi-agent reinforcement learning model of common-pool resource appropriationcited by 69
  23. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  24. 2020Social diversity and social preferences in mixed-motive reinforcement learningcited by 49
  25. 2017Maintaining cooperation in complex social dilemmas using deep reinforcement learningcited by 48
  26. 2017Prosocial learning agents solve generalized Stag Hunts better than selfish onescited by 47
  27. 2021Podracer architectures for scalable Reinforcement Learningcited by 38
  28. 2020Too many cooks: Bayesian inference for coordinating multi-agent collaborationcited by 29
  29. 2021A learning agent that acquires social norms from public sanctions in decentralized multi-agent settingscited by 24
  30. 2022Emergent Bartering Behaviour in Multi-Agent Reinforcement Learningcited by 19
  31. 2018Malthusian Reinforcement Learningcited by 14
  32. 2020Model-free conventions in multi-agent reinforcement learning with heterogeneous preferencescited by 13
  33. 2021Modelling Cooperation in Network Games with Spatio-Temporal Complexitycited by 5
  34. 2021Statistical discrimination in learning agentscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 11; 2025: 9; 2026: 14.

40 papers.

Sort:

By area:

  1. 2023JaxMARL: Multi-Agent RL Environments and Algorithms in JAXcited by 63
  2. 2024Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Modelscited by 48
  3. 2023A Review of Cooperation in Multi-agent Learningcited by 29
  4. 2024POGEMA: A Benchmark Platform for Cooperative Multi-Agent Pathfindingcited by 21
  5. 2025Virtual Agent Economiescited by 21
  6. 2024AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Makingcited by 20
  7. 2024N-Agent Ad Hoc Teamworkcited by 15
  8. 2025Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordiacited by 12
  9. 2024Can LLM-Augmented autonomous agents cooperate?, An evaluation of their cooperative capabilities through Melting Potcited by 11
  10. 2025Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culturecited by 11
  11. 2025MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learningcited by 11
  12. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
28 more
  1. 2024Cooperative Resilience in Artificial Intelligence Multiagent Systemscited by 8
  2. 2025SocialJax: An Evaluation Suite for Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 8
  3. 2025CoopReflect: Towards Natural Language Communication for Cooperative Autonomous Driving via Multi-Agent Learningcited by 7
  4. 2025Evolution of Cooperation in LLM-Agent Societies: A Preliminary Study Using Different Punishment Strategiescited by 6
  5. 2026Generative AI collective behavior needs an interactionist paradigmcited by 6
  6. 2024CoDreamer: Communication-Based Decentralised World Modelscited by 5
  7. 2026ActionParty: Multi-Subject Action Binding in Generative Video Gamescited by 5
  8. 2024Multi-agent cooperation through learning-aware policy gradientscited by 4
  9. 2025Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibrationcited by 4
  10. 2026Metric-Gradient Projection for Stable Multi-Agent Policy Learningcited by 4
  11. 2023Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmascited by 2
  12. 2023Doing the right thing for the right reason: Evaluating artificial moral cognition by probing cost insensitivitycited by 2
  13. 2023Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalizationcited by 2
  14. 2024Social Environment Designcited by 2
  15. 2024KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordinationcited by 2
  16. 2026Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamworkcited by 2
  17. 2026Benchmarking Open-Ended Multi-Agent Coordination in Language Agentscited by 2
  18. 2023marl-jax: Multi-Agent Reinforcement Leaning Frameworkcited by 1
  19. 2024Democratizing Reward Design for Personal and Representative Value-Alignmentcited by 1
  20. 2026JaxAHT: A JAX-Based Library for Ad Hoc Teamworkcited by 1
  21. 2026Theory of Scene: Breaking the Symmetry Trap in Multi-Agent LLM Coordinationcited by 1
  22. 2025Bidirectional Distillation: A Mixed-Play Framework for Multi-Agent Generalizable Behaviorscited by 0
  23. 2026Learning Incentive Structures for Cooperative Resilience in Multi-Agent Systems under Social Dilemmascited by 0
  24. 2026Strategically Robust Multi-Agent Reinforcement Learning with Linear Function Approximationcited by 0
  25. 2026CoGrid & the Multi-User Gymnasium: A Framework for Multi-Agent Experimentationcited by 0
  26. 2026Who Bears the Burden? Learning Responsibility for Shared Constraints in Multi-Agent Reinforcement Learningcited by 0
  27. 2026BluffJAX: Adversarial Imperfect Information Games in JAXcited by 0
  28. 2026Partially Observable Zero-shot coordination by Predicting Intention of Partnercited by 0