# Distributional Reward Decomposition for Reinforcement Learning

arXiv: https://arxiv.org/abs/1911.02166 · HTML: https://mlsanity.com/paper/1911.02166/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2019.

## Built on by (most cited first, 11 of 11)

- [A Survey on Transformers in Reinforcement Learning](https://mlsanity.com/paper/2301.03044/) (2023)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learning](https://mlsanity.com/paper/2102.07936/) (2021)
- [Distributional Reinforcement Learning for Multi-Dimensional Reward Functions](https://mlsanity.com/paper/2110.13578/) (2021)
- [Distributional Reinforcement Learning with Regularized Wasserstein Loss](https://mlsanity.com/paper/2202.00769/) (2022)
- [People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Explainable AI](https://mlsanity.com/paper/2403.08828/) (2024)
- [A Unified Framework for Factorizing Distributional Value Functions for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.02430/) (2023)
- [From Actions to Words: Towards Abstractive-Textual Policy Summarization in RL](https://mlsanity.com/paper/2503.10509/) (2025)
- [Beyond Scalar Rewards: Distributional Reinforcement Learning with Preordered Objectives for Safe and Reliable Autonomous Driving](https://mlsanity.com/paper/2603.20230/) (2026)
- [Calibration-First Reward-Component Auditing for Reinforcement Learning Control in Smart Greenhouses](https://mlsanity.com/paper/2607.11959/) (2026)
- [Automated Testing of LLM-Based Post Hoc Explainers Using Model Checking as an Oracle](https://mlsanity.com/paper/2608.30581/) (2026)

## Abstract (arXiv)

Many reinforcement learning (RL) tasks have specific properties that can be leveraged to modify existing RL algorithms to adapt to those tasks and further improve performance, and a general class of such properties is the multiple reward channel. In those environments the full reward can be decomposed into sub-rewards obtained from different channels. Existing work on reward decomposition either requires prior knowledge of the environment to decompose the full reward, or decomposes reward without prior knowledge but with degraded performance. In this paper, we propose Distributional Reward Decomposition for Reinforcement Learning (DRDRL), a novel reward decomposition algorithm which captures the multiple reward channel structure under distributional setting. Empirically, our method captures the multi-channel structure and discovers meaningful reward decomposition, without any requirements on prior knowledge. Consequently, our agent achieves better performance than existing methods on environments with multiple reward channels.