mlsanityData to 7 Oct 2026

Opponent Modeling in Deep Reinforcement Learning

arXiv 1609.05559 ↗First posted 18 Sep 2016cs.LGcited by 94

Not on the map. Why?

Abstract (arXiv)

Opponent modeling is necessary in multi-agent settings where secondary agents with competing goals also adapt their strategies, yet it remains challenging because strategies interact with each other and change. Most previous work focuses on developing probabilistic models or parameterized strategies for specific applications. Inspired by the recent success of deep reinforcement learning, we present neural-based models that jointly learn a policy and the behavior of opponents. Instead of explicitly predicting the opponent's action, we encode observation of the opponents into a deep Q-Network (DQN); however, we retain explicit modeling (if desired) using multitasking. By using a Mixture-of-Experts architecture, our model automatically discovers different strategy patterns of opponents without extra supervision. We evaluate our models on a simulated soccer game and a popular trivia game, showing superior performance over DQN and its variants.

Key facts. Cited by 94 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 0; 2019: 0; 2020: 0; 2021: 16; 2022: 21; 2023: 19; 2024: 10; 2025: 18; 2026: 9.

94 papers.

Sort:

By area:

  1. 2022A Survey on Model-based Reinforcement Learningcited by 65
  2. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 64
  3. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  4. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  5. 2021Deep Multiagent Reinforcement Learning: Challenges and Directionscited by 42
  6. 2022COLA: Consistent Learning with Opponent-Learning Awarenesscited by 38
  7. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  8. 2023Dynamic Handover: Throw and Catch with Bimanual Handscited by 33
  9. 2021Deep Interactive Bayesian Reinforcement Learning via Meta-Learningcited by 29
  10. 2023PECAN: Leveraging Policy Ensemble for Context-Aware Zero-Shot Human-AI Coordinationcited by 25
  11. 2021Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rolloutscited by 24
  12. 2021Influencing Towards Stable Multi-Agent Interactionscited by 24
38 more
  1. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  2. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  3. 2021Model-Based Opponent Modelingcited by 21
  4. 2023Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learningcited by 19
  5. 2022Influencing Long-Term Behavior in Multiagent Reinforcement Learningcited by 18
  6. 2022Coordination with Humans via Strategy Matchingcited by 14
  7. 2021LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammatescited by 9
  8. 2022DouZero+: Improving DouDizhu AI by Opponent Modeling and Coach-guided Learningcited by 7
  9. 2022TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimizationcited by 7
  10. 2023DanZero+: Dominating the GuanDan Game through Reinforcement Learningcited by 6
  11. 2024The Curse of Diversity in Ensemble-Based Explorationcited by 6
  12. 2024Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowdscited by 6
  13. 2021Investigation of Independent Reinforcement Learning Algorithms in Multi-Agent Environmentscited by 5
  14. 2022DanZero: Mastering GuanDan Game with Reinforcement Learningcited by 5
  15. 2023iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learningcited by 5
  16. 2023Learning Models of Adversarial Agent Behavior under Partial Observabilitycited by 5
  17. 2023(Ir)rationality in AI: State of the Art, Research Challenges and Open Questionscited by 5
  18. 2025GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibrationcited by 5
  19. 2021Multi-Agent Advisor Q-Learningcited by 4
  20. 2023Co-Learning Empirical Games and World Modelscited by 4
  21. 2023Attention Schema in Neural Agentscited by 4
  22. 2024Human-Agent Coordination in Games under Incomplete Information via Multi-Step Intentcited by 4
  23. 2024Predicting Future Actions of Reinforcement Learning Agentscited by 4
  24. 2025Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)cited by 4
  25. 2021DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulationcited by 3
  26. 2022Motivating Physical Activity via Competitive Human-Robot Interactioncited by 3
  27. 2023CAMMARL: Conformal Action Modeling in Multi Agent Reinforcement Learningcited by 3
  28. 2025Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Explorationcited by 3
  29. 2025VGC-Bench: Towards Mastering Diverse Team Strategies in Competitive Pok'emoncited by 3
  30. 2025Skill-Aligned Fairness in Multi-Agent Learning for Collaboration in Healthcarecited by 3
  31. 2021Information-Bottleneck-Based Behavior Representation Learning for Multi-agent Reinforcement learningcited by 2
  32. 2021Learning to Cooperate with Unseen Agent via Meta-Reinforcement Learningcited by 2
  33. 2022Recursive Reasoning Graph for Multi-Agent Reinforcement Learningcited by 2
  34. 2022BRExIt: On Opponent Modelling in Expert Iterationcited by 2
  35. 2022Offline Equilibrium Findingcited by 2
  36. 2022Centralized Training with Hybrid Execution in Multi-Agent Reinforcement Learningcited by 2
  37. 2022Decision-making with Speculative Opponent Modelscited by 2
  38. 2023Adversarial Search and Tracking with Multiagent Reinforcement Learning in Sparsely Observable Environmentcited by 2

50 most cited of 94 All 94 (JSON)