mlsanityData to 7 Oct 2026

Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective

arXiv 2106.05402 ↗First posted 9 Jun 2021cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Within the framework of Multi-Agent Reinforcement Learning, Social Learning is a new class of algorithms that enables agents to reshape the reward function of other agents with the goal of promoting cooperation and achieving higher global rewards in mixed-motive games. However, this new modification allows agents unprecedented access to each other's learning process, which can drastically increase the risk of manipulation when an agent does not realize it is being deceived into adopting policies which are not actually in its own best interest. This research review introduces the problem statement, defines key concepts, critically evaluates existing evidence and addresses open problems that should be addressed in future research.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 39 of its 79 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

39 of 79 references.

References · 39

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  4. 2019Solving Rubik's Cube with a Robot Handcited by 699
  5. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  6. 2018Reinforcement Learning for Solving the Vehicle Routing Problemcited by 327
  7. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  8. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
  9. 2020"Other-Play" for Zero-Shot Coordinationcited by 194
  10. 2020Open Problems in Cooperative AIcited by 167
  11. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  12. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
27 more
  1. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 107
  2. 2018On Learning Intrinsic Rewards for Policy Gradient Methodscited by 100
  3. 2017A Survey of Learning in Multiagent Environments: Dealing with Non-Stationaritycited by 93
  4. 2019Deep Reinforcement Learning for Cyber Securitycited by 92
  5. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 90
  6. 2020The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policiescited by 75
  7. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  8. 2017A multi-agent reinforcement learning model of common-pool resource appropriationcited by 68
  9. 2020Learning to Incentivize Other Learning Agentscited by 55
  10. 2020Social diversity and social preferences in mixed-motive reinforcement learningcited by 49
  11. 2019Leveraging Human Guidance for Deep Reinforcement Learning Taskscited by 42
  12. 2019Environment Probing Interaction Policiescited by 39
  13. 2020On the Robustness of Cooperative Multi-Agent Reinforcement Learningcited by 39
  14. 2018CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learningcited by 31
  15. 2019Theory of Minds: Understanding Behavior in Groups Through Inverse Planningcited by 30
  16. 2017Consequentialist conditional cooperation in social dilemmas with imperfect informationcited by 23
  17. 2019Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advancescited by 23
  18. 2019A New Framework for Multi-Agent Reinforcement Learning -- Centralized Training and Exploration with Decentralized Execution via Policy Distillationcited by 12
  19. 2019Robust Opponent Modeling via Adversarial Ensemble Reinforcement Learning in Asymmetric Imperfect-Information Gamescited by 11
  20. 2018Adaptive Mechanism Design: Learning to Promote Cooperationcited by 9
  21. 2019Convergence of Multi-Agent Learning with a Finite Step Size in General-Sum Gamescited by 8
  22. 2020Multi-Agent Reinforcement Learning for Unmanned Aerial Vehicle Coordination by Multi-Critic Policy Gradient Optimizationcited by 5
  23. 2016Bayesian Opponent Exploitation in Imperfect-Information Gamescited by 4
  24. 2019Superstition in the Network: Deep Reinforcement Learning Plays Deceptive Gamescited by 4
  25. 2018Cooperation Enforcement and Collusion Resistance in Repeated Public Goods Gamescited by 1
  26. 2020Inducing Cooperative behaviour in Sequential-Social dilemmas through Multi-Agent Reinforcement Learning using Status-Quo Losscited by 1
  27. 2020Learning Complex Multi-Agent Policies in Presence of an Adversarycited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 0; 2024: 1; 2025: 1.

4 papers.

  1. 2022AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-Ncited by 22
  2. 2025Scheming Ability in LLM-to-LLM Strategic Interactionscited by 4
  3. 2024Distributed Online Life-Long Learning (DOL3) for Multi-agent Trust and Reputation Assessment in E-commercecited by 1
  4. 2022A Cooperative Reinforcement Learning Environment for Detecting and Penalizing Betrayalcited by 0