# Regret Minimization for Partially Observable Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1710.11424 · HTML: https://mlsanity.com/paper/1710.11424/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 12 of 12)

- [A Survey on Self-play Methods in Reinforcement Learning](https://mlsanity.com/paper/2408.01072/) (2024)
- [Accelerating the Learning of TAMER with Counterfactual Explanations](https://mlsanity.com/paper/2108.01358/) (2021)
- [ReMIX: Regret Minimization for Monotonic Value Function Factorization in Multiagent Reinforcement Learning](https://mlsanity.com/paper/2302.05593/) (2023)
- [Robust Deep Reinforcement Learning against Adversarial Behavior Manipulation](https://mlsanity.com/paper/2406.03862/) (2024)
- [On Minimizing Adversarial Counterfactual Error in Adversarial RL](https://mlsanity.com/paper/2406.04724/) (2024)
- [Unified Policy Optimization for Continuous-action Reinforcement Learning in Non-stationary Tasks and Games](https://mlsanity.com/paper/2208.09452/) (2022)
- [Action Shapley: A Training Data Selection Metric for World Model in Reinforcement Learning](https://mlsanity.com/paper/2601.10905/) (2026)
- [Optimize Neural Fictitious Self-Play in Regret Minimization Thinking](https://mlsanity.com/paper/2104.10845/) (2021)
- [Dual Behavior Regularized Reinforcement Learning](https://mlsanity.com/paper/2109.09037/) (2021)
- [Game-theoretic Objective Space Planning](https://mlsanity.com/paper/2209.07758/) (2022)
- [Reinforcement Learning of Display Transfer Robots in Glass Flow Control Systems: A Physical Simulation-Based Approach](https://mlsanity.com/paper/2310.07981/) (2023)
- [Dopamine-driven synaptic credit assignment in neural networks](https://mlsanity.com/paper/2510.22178/) (2025)

## Abstract (arXiv)

Deep reinforcement learning algorithms that estimate state and state-action value functions have been shown to be effective in a variety of challenging domains, including learning control strategies from raw image pixels. However, algorithms that estimate state and state-action value functions typically assume a fully observed state and must compensate for partial observations by using finite length observation histories or recurrent networks. In this work, we propose a new deep reinforcement learning algorithm based on counterfactual regret minimization that iteratively updates an approximation to an advantage-like function and is robust to partially observed state. We demonstrate that this new algorithm can substantially outperform strong baseline methods on several partially observed reinforcement learning tasks: learning first-person 3D navigation in Doom and Minecraft, and acting in the presence of partially observed objects in Doom and Pong.