mlsanityData to 7 Oct 2026

Imitation Learning with Concurrent Actions in 3D Games

arXiv 1803.05402 ↗First posted 14 Mar 2018cs.AI cs.LG stat.MLcited by 18

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

In this work we describe a novel deep reinforcement learning architecture that allows multiple actions to be selected at every time-step in an efficient manner. Multi-action policies allow complex behaviours to be learnt that would otherwise be hard to achieve when using single action selection techniques. We use both imitation learning and temporal difference (TD) reinforcement learning (RL) to provide a 4x improvement in training time and 2.5x improvement in performance over single action selection TD RL. We demonstrate the capabilities of this network using a complex in-house 3D game. Mimicking the behavior of the expert teacher significantly improves world state exploration and allows the agents vision system to be trained more rapidly than TD RL alone. This initial training technique kick-starts TD learning and the agent quickly learns to surpass the capabilities of the expert.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 28 references that are in the map. First posted on arXiv 14 Mar 2018.

Builds on

16 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Reinforcement Learning from Imperfect Demonstrations14 Feb

Other references · 15

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  8. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  9. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  10. 2017Noisy Networks for Explorationcited by 262
  11. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  12. 2017Deep Q-learning from Demonstrationscited by 27
3 more
  1. 2017Learning to Factor Policies and Action-Value Functions: Factored Action Space Representations for Deep Reinforcement learningcited by 15
  2. 2018Pretraining Deep Actor-Critic Reinforcement Learning Algorithms With Expert Demonstrationscited by 6
  3. 2018Learning High-level Representations from Demonstrationscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 4; 2022: 1; 2023: 3; 2024: 5; 2025: 3; 2026: 1.

18 papers.

Sort:

By area:

  1. 2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 49
  2. 2021Augmenting Automated Game Testing with Deep Reinforcement Learningcited by 38
  3. 2023Diffusion Model-Augmented Behavioral Cloningcited by 37
  4. 2021Adversarial Reinforcement Learning for Procedural Content Generationcited by 36
  5. 2024Diffusion-Reward Adversarial Imitation Learningcited by 18
  6. 2020Benchmarking End-to-End Behavioural Cloning on Video Gamescited by 15
  7. 2022Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learningcited by 13
  8. 2023Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Gamescited by 12
  9. 2024Diffusion Imitation from Observationcited by 12
  10. 2024Efficient Active Imitation Learning with Random Network Distillationcited by 7
  11. 2023Improving Generalization in Game Agents with Data Augmentation in Imitation Learningcited by 4
  12. 2021Mimicking Playstyle by Adapting Parameterized Behavior Trees in RTS Gamescited by 1
6 more
  1. 2024Carefully Structured Compression: Efficiently Managing StarCraft II Datacited by 1
  2. 2024Human-like Bots for Tactical Shooters Using Compute-Efficient Sensorscited by 1
  3. 2025Learning Representations in Video Game Agents with Supervised Contrastive Imitation Learningcited by 1
  4. 2025Restoring Noisy Demonstration for Imitation Learning With Diffusion Modelscited by 1
  5. 2026Improving Regret Approximation for Unsupervised Dynamic Environment Generationcited by 1
  6. 2025An Improved Multi-Agent Algorithm for Cooperative and Competitive Environments by Identifying and Encouraging Cooperation among Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025An Introduction to Deep Reinforcement and Imitation Learning
  2. 2025Going Beyond Expert Performance via Deep Implicit Imitation Reinforcement Learning
  3. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning
  4. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks
  5. 2025Adapting a World Model for Trajectory Following in a 3D Game
  6. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  7. 2018Multi-Agent Generative Adversarial Imitation Learning
  8. 2024Efficient Imitation Learning with Conservative World Models