Data to 7 Oct 2026

Playing FPS Games with Deep Reinforcement Learning

arXiv 1609.05521 ↗First posted 18 Sep 2016cs.AI cs.LGcited by 95

Not on the map. Why?

Abstract (arXiv)

Advances in deep reinforcement learning have allowed autonomous agents to perform well on Atari games, often outperforming humans, using only raw pixels to make their decisions. However, most of these games take place in 2D environments that are fully observable to the agent. In this paper, we present the first architecture to tackle 3D environments in first-person shooter games, that involve partially observable states. Typically, deep reinforcement learning methods only utilize visual input for training. We present a method to augment these models to exploit game feature information such as the presence of enemies or items, during the training phase. Our model is trained to simultaneously learn these features along with minimizing a Q-learning objective, which is shown to dramatically improve the training speed and performance of our agent. Our architecture is also modularized to allow different models to be independently trained for different phases of the game. We show that the proposed architecture substantially outperforms built-in AI agents of the game as well as humans in deathmatch scenarios.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 20; 2023: 15; 2024: 17; 2025: 11; 2026: 7.

95 papers.

Sort:

By area:

2022Masked Visual Pre-training for Motor Controlcited by 246
2022Navigating to Objects in the Real Worldcited by 104
2021Temporal Predictive Coding For Model-Based Planning In Latent Spacecited by 54
2021Memory-based Deep Reinforcement Learning for POMDPscited by 53
2021Counter-Strike Deathmatch with Large-Scale Behavioural Cloningcited by 49
2022Retrospectives on the Embodied AI Workshopcited by 46
2021Augmenting Automated Game Testing with Deep Reinforcement Learningcited by 38
2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
2022Deep Transformer Q-Networks for Partially Observable Reinforcement Learningcited by 35
2022Robust Task Representations for Offline Meta-Reinforcement Learning via Contrastive Learningcited by 33
2022Chain of Thought Imitation with Procedure Cloningcited by 30
2025SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Predictioncited by 26

38 more

2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
2021Recurrent Off-policy Baselines for Memory-based Continuous Controlcited by 19
2021Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigationcited by 17
2021Branching Time Active Inference: the theory and its generalitycited by 15
2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13
2022A Novel Deep Reinforcement Learning Based Automated Stock Trading System Using Cascaded LSTM Networkscited by 12
2023Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothnesscited by 12
2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 12
2021Robotic Surgery With Lean Reinforcement Learningcited by 11
2022Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomescited by 11
2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
2021Phy-Q as a measure for physical reasoning intelligencecited by 10
2023Navigates Like Me: Understanding How People Evaluate Human-Like AI in Video Gamescited by 10
2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
2021Branching Time Active Inference: empirical study and complexity class analysiscited by 9
2022Online Learning of Reusable Abstract Models for Object Goal Navigationcited by 9
2023Towards Source-free Domain Adaptive Semantic Segmentation via Importance-aware and Prototype-contrast Learningcited by 9
2025A Comprehensive Review of Multi-Agent Reinforcement Learning in Video Gamescited by 9
2021SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learningcited by 7
2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policiescited by 7
2024ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learningcited by 7
2021Towards Disturbance-Free Visual Mobile Manipulationcited by 6
2022Explainability Via Causal Self-Talkcited by 6
2024Will GPT-4 Run DOOM?cited by 6
2021MEPG: A Minimalist Ensemble Policy Gradient Framework for Deep Reinforcement Learningcited by 5
2021Graph augmented Deep Reinforcement Learning in the GameRLand3D environmentcited by 5
2023Deconstructing deep active inferencecited by 5
2023Robust Driving Policy Learning with Guided Meta Reinforcement Learningcited by 5
2023A Survey on Service Route and Time Prediction in Instant Delivery: Taxonomy, Progress, and Prospectscited by 5
2024LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agentscited by 5
2024Training Interactive Agent in Large FPS Game Map with Rule-enhanced Reinforcement Learningcited by 5
2024Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Makingcited by 5
2021Lifetime policy reuse and the importance of task capacitycited by 4
2021Godot Reinforcement Learning Agentscited by 4
2023Noise Distribution Decomposition based Multi-Agent Distributional Reinforcement Learningcited by 4
2024Learning to Move Like Professional Counter-Strike Playerscited by 4
2021Reward prediction for representation learning and reward shapingcited by 3
2021Distilling Reinforcement Learning Tricks for Video Gamescited by 3

50 most cited of 95 All 95 (JSON)