mlsanityData to 7 Oct 2026

Learning Strategy Representation for Imitation Learning in Multi-Agent Games

arXiv 2409.19363 ↗First posted 28 Sep 2024cs.MA cs.AI cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

The offline datasets for imitation learning (IL) in multi-agent games typically contain player trajectories exhibiting diverse strategies, which necessitate measures to prevent learning algorithms from acquiring undesirable behaviors. Learning representations for these trajectories is an effective approach to depicting the strategies employed by each demonstrator. However, existing learning strategies often require player identification or rely on strong assumptions, which are not appropriate for multi-agent games. Therefore, in this paper, we introduce the Strategy Representation for Imitation Learning (STRIL) framework, which (1) effectively learns strategy representations in multi-agent games, (2) estimates proposed indicators based on these representations, and (3) filters out sub-optimal data using the indicators. STRIL is a plug-in method that can be integrated into existing IL algorithms. We demonstrate the effectiveness of STRIL across competitive multi-agent scenarios, including Two-player Pong, Limit Texas Hold'em, and Connect Four. Our approach successfully acquires strategy representations and indicators, thereby identifying dominant trajectories and significantly enhancing existing IL performance across these environments.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 25 of its 34 references that are in the map. First posted on arXiv 28 Sep 2024.

Builds on

25 of 34 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Imitation Learning by Estimating Expertise of Demonstrators2 Feb
  2. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Data27 Oct
  3. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality27 Oct
  4. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  5. 2020Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets19 May
  6. 2019Goal-conditioned Imitation Learning13 Jun

Other references · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  7. 2016End to End Learning for Self-Driving Carscited by 1,000
  8. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  9. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  10. 2019Learning Latent Plans from Playcited by 310
  11. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  12. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
7 more
  1. 2017Emergent Complexity via Multi-Agent Competitioncited by 117
  2. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  3. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regressioncited by 84
  4. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  5. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  6. 2018Learning Policy Representations in Multiagent Systemscited by 48
  7. 2024Select to Perfect: Imitating desired behavior from large multi-agent datacited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Mimicking To Dominate: Imitation Learning Strategies for Success in Multiagent Competitive Games
  2. 2022Conditional Imitation Learning for Multi-Agent Games
  3. 2025Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning
  4. 2022A Ranking Game for Imitation Learning
  5. 2024Efficient Active Imitation Learning with Random Network Distillation
  6. 2025Structured Imitation Learning of Interactive Policies through Inverse Games
  7. 2018Multi-Agent Generative Adversarial Imitation Learning
  8. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances