# Convergence Guarantees for Deep Epsilon Greedy Policy Learning

arXiv: https://arxiv.org/abs/2112.03376 · HTML: https://mlsanity.com/paper/2112.03376/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 13 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 4 of 4)

- [On Lipschitz Bounds of General Convolutional Neural Networks](https://mlsanity.com/paper/1808.01415/) (2018)
- [Doubly Robust Off-Policy Actor-Critic: Convergence and Optimality](https://mlsanity.com/paper/2102.11866/) (2021)
- [Doubly Robust Off-Policy Learning on Low-Dimensional Manifolds by Deep Neural Networks](https://mlsanity.com/paper/2011.01797/) (2020)
- [Deep Upper Confidence Bound Algorithm for Contextual Bandit Ranking of Information Selection](https://mlsanity.com/paper/2110.04127/) (2021)

## Built on by (most cited first, 6 of 6)

- [Optimal Policy Learning with Observational Data in Multi-Action Scenarios: Estimation, Risk Preference, and Potential Failures](https://mlsanity.com/paper/2403.20250/) (2024)
- [A Generative Framework for Predictive Modeling of Multiple Chronic Conditions Using Graph Variational Autoencoder and Bandit-Optimized Graph Neural Network](https://mlsanity.com/paper/2409.13671/) (2024)
- [Toward Debugging Deep Reinforcement Learning Programs with RLExplorer](https://mlsanity.com/paper/2410.04322/) (2024)
- [Optimal Transport for Super Resolution Applied to Astronomy Imaging](https://mlsanity.com/paper/2202.05354/) (2022)
- [A Deep Reinforcement Learning Approach for Interactive Search with Sentence-level Feedback](https://mlsanity.com/paper/2310.03043/) (2023)
- [Bandit Guided Submodular Curriculum for Adaptive Subset Selection](https://mlsanity.com/paper/2511.22944/) (2025)

## Abstract (arXiv)

Policy learning is a quickly growing area. As robotics and computers control day-to-day life, their error rate needs to be minimized and controlled. There are many policy learning methods and bandit methods with provable error rates that accompany them. We show an error or regret bound and convergence of the Deep Epsilon Greedy method which chooses actions with a neural network's prediction. We also show that Epsilon Greedy method regret upper bound is minimized with cubic root exploration. In experiments with the real-world dataset MNIST, we construct a nonlinear reinforcement learning problem. We witness how with either high or low noise, some methods do and some do not converge which agrees with our proof of convergence.