# QUOTA: The Quantile Option Architecture for Reinforcement Learning

arXiv: https://arxiv.org/abs/1811.02073 · HTML: https://mlsanity.com/paper/1811.02073/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2018.

## Built on by (most cited first, 13 of 13)

- [Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions](https://mlsanity.com/paper/2112.11561/) (2021)
- [Tactical Optimism and Pessimism for Deep Reinforcement Learning](https://mlsanity.com/paper/2102.03765/) (2021)
- [DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learning](https://mlsanity.com/paper/2102.07936/) (2021)
- [Exploring the Training Robustness of Distributional Reinforcement Learning against Noisy State Observations](https://mlsanity.com/paper/2109.08776/) (2021)
- [The Sufficiency of Off-Policyness and Soft Clipping: PPO is still Insufficient according to an Off-Policy Measure](https://mlsanity.com/paper/2205.10047/) (2022)
- [Damped Anderson Mixing for Deep Reinforcement Learning: Acceleration, Convergence, and Stabilization](https://mlsanity.com/paper/2110.08896/) (2021)
- [Towards Safe, Explainable, and Regulated Autonomous Driving](https://mlsanity.com/paper/2111.10518/) (2021)
- [Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control](https://mlsanity.com/paper/2302.11694/) (2023)
- [Introspective Experience Replay: Look Back When Surprised](https://mlsanity.com/paper/2206.03171/) (2022)
- [Pitfall of Optimism: Distributional Reinforcement Learning by Randomizing Risk Criterion](https://mlsanity.com/paper/2310.16546/) (2023)
- [Interpretable Stochastic Model Predictive Control using Distributional Reinforced Estimation for Quadrotor Tracking Systems](https://mlsanity.com/paper/2205.07150/) (2022)
- [A Unified Framework for Factorizing Distributional Value Functions for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.02430/) (2023)
- [DRL-ORA: Distributional Reinforcement Learning with Online Risk Adaption](https://mlsanity.com/paper/2310.05179/) (2023)

## Abstract (arXiv)

In this paper, we propose the Quantile Option Architecture (QUOTA) for exploration based on recent advances in distributional reinforcement learning (RL). In QUOTA, decision making is based on quantiles of a value distribution, not only the mean. QUOTA provides a new dimension for exploration via making use of both optimism and pessimism of a value distribution. We demonstrate the performance advantage of QUOTA in both challenging video games and physical robot simulators.