# A Comparative Study of Deep Reinforcement Learning Models: DQN vs PPO vs A2C

arXiv: https://arxiv.org/abs/2407.14151 · HTML: https://mlsanity.com/paper/2407.14151/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 8 references that are in the map.
- First posted on arXiv 19 Jul 2024.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)

## Built on by (most cited first, 8 of 8)

- [Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents](https://mlsanity.com/paper/2509.16151/) (2025)
- [Context-Aware Adaptive Sampling for Intelligent Data Acquisition Systems Using DQN](https://mlsanity.com/paper/2504.09344/) (2025)
- [Using Machine Learning to Take Stay-or-Go Decisions in Data-driven Drone Missions](https://mlsanity.com/paper/2512.04773/) (2025)
- [Component Centric Placement Using Deep Reinforcement Learning](https://mlsanity.com/paper/2602.23540/) (2026)
- [Safe reinforcement learning with online filtering for fatigue-predictive human-robot task planning and allocation in production](https://mlsanity.com/paper/2604.12667/) (2026)
- [A hierarchical spatial-aware algorithm with efficient reinforcement learning for human-robot task planning and allocation in production](https://mlsanity.com/paper/2604.12669/) (2026)
- [Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings](https://mlsanity.com/paper/2606.02049/) (2026)
- [Trading Confidence: Comprehensive Uncertainty Estimation in Algorithmic Trading](https://mlsanity.com/paper/2607.02864/) (2026)

## Abstract (arXiv)

This study conducts a comparative analysis of three advanced Deep Reinforcement Learning models: Deep Q-Networks (DQN), Proximal Policy Optimization (PPO), and Advantage Actor-Critic (A2C), within the BreakOut Atari game environment. Our research assesses the performance and effectiveness of these models in a controlled setting. Through rigorous experimentation, we examine each model's learning efficiency, strategy development, and adaptability under dynamic game conditions. The findings provide critical insights into the practical applications of these models in game-based learning environments and contribute to the broader understanding of their capabilities. The code is publicly available at github.com/Neilus03/DRL_comparative_study.