# Enhancing Two-Player Performance Through Single-Player Knowledge Transfer: An Empirical Study on Atari 2600 Games

arXiv: https://arxiv.org/abs/2410.16653 · HTML: https://mlsanity.com/paper/2410.16653/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 20 of 20)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Improving Sample Efficiency in Model-Free Reinforcement Learning from Images](https://mlsanity.com/paper/1910.01741/) (2019)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning](https://mlsanity.com/paper/1511.06342/) (2015)
- [Tianshou: a Highly Modularized Deep Reinforcement Learning Library](https://mlsanity.com/paper/2107.14171/) (2021)
- [A Survey of Deep Reinforcement Learning in Video Games](https://mlsanity.com/paper/1912.10944/) (2019)
- [Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation](https://mlsanity.com/paper/1806.07377/) (2018)
- [Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals](https://mlsanity.com/paper/2302.04449/) (2023)
- [Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://mlsanity.com/paper/2009.08120/) (2020)
- [Deep Learning for Video Game Playing](https://mlsanity.com/paper/1708.07902/) (2017)
- [SuperSuit: Simple Microwrappers for Reinforcement Learning Environments](https://mlsanity.com/paper/2008.08932/) (2020)
- [Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Model](https://mlsanity.com/paper/2110.03184/) (2021)
- [Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning](https://mlsanity.com/paper/2105.14780/) (2021)
- [Transformer-based Value Function Decomposition for Cooperative Multi-agent Reinforcement Learning in StarCraft](https://mlsanity.com/paper/2208.07298/) (2022)
- [Marathon Environments: Multi-Agent Continuous Control Benchmarks in a Modern Video Game Engine](https://mlsanity.com/paper/1902.09097/) (2019)
- [On the Verge of Solving Rocket League using Deep Reinforcement Learning and Sim-to-sim Transfer](https://mlsanity.com/paper/2205.05061/) (2022)
- [Task Relabelling for Multi-task Transfer using Successor Features](https://mlsanity.com/paper/2205.10175/) (2022)

## Built on by (most cited first, 1 of 1)

- [A Controlled Study of Double DQN and Dueling DQN Under Cross-Environment Transfer](https://mlsanity.com/paper/2602.09810/) (2026)

## Abstract (arXiv)

Playing two-player games using reinforcement learning and self-play can be challenging due to the complexity of two-player environments and the possible instability in the training process. We propose that a reinforcement learning algorithm can train more efficiently and achieve improved performance in a two-player game if it leverages the knowledge from the single-player version of the same game. This study examines the proposed idea in ten different Atari 2600 environments using the Atari 2600 RAM as the input state. We discuss the advantages of using transfer learning from a single-player training process over training in a two-player setting from scratch, and demonstrate our results in a few measures such as training time and average total reward. We also discuss a method of calculating RAM complexity and its relationship to performance.