# GAN Q-learning

arXiv: https://arxiv.org/abs/1805.04874 · HTML: https://mlsanity.com/paper/1805.04874/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2018.

## Built on by (most cited first, 8 of 8)

- [Distributional Offline Policy Evaluation with Predictive Error Guarantees](https://mlsanity.com/paper/2302.09456/) (2023)
- [Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learning](https://mlsanity.com/paper/2301.12130/) (2023)
- [Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Model](https://mlsanity.com/paper/2402.07598/) (2024)
- [Distributional Bellman Operators over Mean Embeddings](https://mlsanity.com/paper/2312.07358/) (2023)
- [Bayesian Distributional Policy Gradients](https://mlsanity.com/paper/2103.11265/) (2021)
- [Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learning](https://mlsanity.com/paper/2607.08444/) (2026)
- [Online Inference for Quantile Temporal Difference Learning in Distributional Reinforcement Learning](https://mlsanity.com/paper/2608.12973/) (2026)
- [Smooth Q-learning: Accelerate Convergence of Q-learning Using Similarity](https://mlsanity.com/paper/2106.01134/) (2021)

## Abstract (arXiv)

Distributional reinforcement learning (distributional RL) has seen empirical success in complex Markov Decision Processes (MDPs) in the setting of nonlinear function approximation. However, there are many different ways in which one can leverage the distributional approach to reinforcement learning. In this paper, we propose GAN Q-learning, a novel distributional RL method based on generative adversarial networks (GANs) and analyze its performance in simple tabular environments, as well as OpenAI Gym. We empirically show that our algorithm leverages the flexibility and blackbox approach of deep learning models while providing a viable alternative to traditional methods.