mlsanityData to 7 Oct 2026

Inverse reinforcement learning for video games

arXiv 1810.10593 ↗First posted 24 Oct 2018cs.LG cs.AI stat.MLcited by 26

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function. It is often easier to provide demonstrations of a target behavior than to design a reward function describing that behavior. Inverse reinforcement learning (IRL) algorithms can infer a reward from demonstrations in low-dimensional continuous control environments, but there has been little work on applying IRL to high-dimensional video games. In our CNN-AIRL baseline, we modify the state-of-the-art adversarial IRL (AIRL) algorithm to use CNNs for the generator and discriminator. To stabilize training, we normalize the reward and increase the size of the discriminator training dataset. We additionally learn a low-dimensional state representation using a novel autoencoder architecture tuned for video game environments. This embedding is used as input to the reward network, improving the sample efficiency of expert demonstrations. Our method achieves high-level performance on the simple Catcher video game, substantially outperforming the CNN-AIRL baseline. We also score points on the Enduro Atari racing game, but do not match expert performance, highlighting the need for further work.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 36 references that are in the map. First posted on arXiv 24 Oct 2018.

Builds on

19 of 36 references.

References · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  5. 2016End to End Learning for Self-Driving Carscited by 1,000
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  8. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  9. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  10. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  11. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  12. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
7 more
  1. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  2. 2017Third-Person Imitation Learningcited by 103
  3. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flowcited by 103
  4. 2018Playing hard exploration games by watching YouTubecited by 98
  5. 2018Learning Plannable Representations with Causal InfoGANcited by 56
  6. 2017The Atari Grand Challenge Datasetcited by 14
  7. 2013Image Segmentation in Video Sequences: A Probabilistic Approachcited by 5

Built on by

Cited by year

Citing papers per year: 2019: 2; 2020: 5; 2021: 1; 2022: 3; 2023: 8; 2024: 4; 2025: 3.

26 papers.

Sort:

By area:

  1. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  2. 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learningcited by 53
  3. 2020Intrinsic Reward Driven Imitation Learning via Generative Modelcited by 35
  4. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
  5. 2022A Primer on Maximum Causal Entropy Inverse Reinforcement Learningcited by 13
  6. 2022Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learningcited by 13
  7. 2023Massively Scalable Inverse Reinforcement Learning in Google Mapscited by 11
  8. 2023Explore 3D Dance Generation via Reward Model from Automatically-Ranked Demonstrationscited by 8
  9. 2022New Challenges in Reinforcement Learning: A Survey of Security and Privacycited by 7
  10. 2019Imitation Learning from Pixel-Level Demonstrations by HashRewardcited by 6
  11. 2020Efficiently Guiding Imitation Learning Agents with Human Gazecited by 6
  12. 2020Demonstration-efficient Inverse Reinforcement Learning in Procedurally Generated Environmentscited by 6
14 more
  1. 2021Policy Fusion for Adaptive and Customizable Reinforcement Learning Agentscited by 4
  2. 2023Improving Generalization in Game Agents with Data Augmentation in Imitation Learningcited by 4
  3. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shapingcited by 3
  4. 2023Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challengescited by 2
  5. 2025Inverse Reinforcement Learning for Minimum-Exposure Paths in Spatiotemporally Varying Scalar Fieldscited by 1
  6. 2023Active Vision Reinforcement Learning under Limited Visual Observabilitycited by 0
  7. 2023Can Differentiable Decision Trees Enable Interpretable Reward Learning from Human Feedback?cited by 0
  8. 2023Replication of Multi-agent Reinforcement Learning for the "Hide and Seek" Problemcited by 0
  9. 2023Distance-rank Aware Sequential Reward Learning for Inverse Reinforcement Learning with Sub-optimal Demonstrationscited by 0
  10. 2024Visual Imitation Learning with Calibrated Contrastive Representationcited by 0
  11. 2024A Model-Based Approach for Improving Reinforcement Learning Efficiency Leveraging Expert Observationscited by 0
  12. 2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proofcited by 0
  13. 2025Neurodivergent Influenceability as a Contingent Solution to the AI Alignment Problemcited by 0
  14. 2025SPLASH! Sample-efficient Preference-based inverse reinforcement learning for Long-horizon Adversarial tasks from Suboptimal Hierarchical demonstrationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Hybrid-AIRL: Enhancing Inverse Reinforcement Learning with Supervised Expert Guidance
  2. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  3. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions
  4. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
  5. 2021Inverse Reinforcement Learning for Strategy Identification
  6. 2025Reinforcement Learning with Inverse Rewards for World Model Post-training
  7. 2019Multi-Agent Adversarial Inverse Reinforcement Learning
  8. 2026Training a Conditioned Video Game Agent on a VLM Annotated Dataset