# Maximum Reward Formulation In Reinforcement Learning

arXiv: https://arxiv.org/abs/2010.03744 · HTML: https://mlsanity.com/paper/2010.03744/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [On the Expressivity of Markov Reward](https://mlsanity.com/paper/2111.00876/) (2021)
- [To the Max: Reinventing Reward in Reinforcement Learning](https://mlsanity.com/paper/2402.01361/) (2024)
- [Reinforcement Learning with Non-Cumulative Objective](https://mlsanity.com/paper/2307.04957/) (2023)
- [Human-AI Collaboration in Real-World Complex Environment with Reinforcement Learning](https://mlsanity.com/paper/2312.15160/) (2023)
- [Learning to Explore in Diverse Reward Settings via Temporal-Difference-Error Maximization](https://mlsanity.com/paper/2506.13345/) (2025)
- [Intelligent O-RAN Traffic Steering for URLLC Through Deep Reinforcement Learning](https://mlsanity.com/paper/2303.01960/) (2023)
- [Tackling Decision Processes with Non-Cumulative Objectives using Reinforcement Learning](https://mlsanity.com/paper/2405.13609/) (2024)
- [Deterministic Exploration via Stationary Bellman Error Maximization](https://mlsanity.com/paper/2410.23840/) (2024)
- [PerfDojo: Automated ML Library Generation for Heterogeneous Architectures](https://mlsanity.com/paper/2511.03586/) (2025)
- [When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?](https://mlsanity.com/paper/2606.18531/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) algorithms typically deal with maximizing the expected cumulative return (discounted or undiscounted, finite or infinite horizon). However, several crucial applications in the real world, such as drug discovery, do not fit within this framework because an RL agent only needs to identify states (molecules) that achieve the highest reward within a trajectory and does not need to optimize for the expected cumulative return. In this work, we formulate an objective function to maximize the expected maximum reward along a trajectory, derive a novel functional form of the Bellman equation, introduce the corresponding Bellman operators, and provide a proof of convergence. Using this formulation, we achieve state-of-the-art results on the task of molecule generation that mimics a real-world drug discovery pipeline.