# Forward-Backward Reinforcement Learning

arXiv: https://arxiv.org/abs/1803.10227 · HTML: https://mlsanity.com/paper/1803.10227/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2018.

## Built on by (most cited first, 19 of 19)

- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [A Survey of Temporal Credit Assignment in Deep Reinforcement Learning](https://mlsanity.com/paper/2312.01072/) (2023)
- [Offline Reinforcement Learning with Reverse Model-based Imagination](https://mlsanity.com/paper/2110.00188/) (2021)
- [PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning](https://mlsanity.com/paper/2106.04152/) (2021)
- [Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imagination](https://mlsanity.com/paper/2206.07989/) (2022)
- [Generative predecessor models for sample-efficient imitation learning](https://mlsanity.com/paper/1904.01139/) (2019)
- [Emergent Dexterity via Diverse Resets and Large-Scale Reinforcement Learning](https://mlsanity.com/paper/2603.15789/) (2026)
- [Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets](https://mlsanity.com/paper/2406.01150/) (2024)
- [Robust Imitation of a Few Demonstrations with a Backwards Model](https://mlsanity.com/paper/2210.09337/) (2022)
- [Solving Sokoban with forward-backward reinforcement learning](https://mlsanity.com/paper/2105.01904/) (2021)
- [Backward Imitation and Forward Reinforcement Learning via Bi-directional Model Rollouts](https://mlsanity.com/paper/2208.02434/) (2022)
- [Backward Learning for Goal-Conditioned Policies](https://mlsanity.com/paper/2312.05044/) (2023)
- [Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning](https://mlsanity.com/paper/2404.04682/) (2024)
- [An Investigation of Time Reversal Symmetry in Reinforcement Learning](https://mlsanity.com/paper/2311.17008/) (2023)
- [ProSpec RL: Plan Ahead, then Execute](https://mlsanity.com/paper/2407.21359/) (2024)
- [Time Reversal Symmetry for Efficient Robotic Manipulations in Deep Reinforcement Learning](https://mlsanity.com/paper/2505.13925/) (2025)
- [Analysis of AI Techniques for Orchestrating Edge-Cloud Application Migration](https://mlsanity.com/paper/2507.10119/) (2025)
- [Inverse Manipulation through Symbolic Planning and Residual Operator Learning](https://mlsanity.com/paper/2606.05248/) (2026)
- [A 3D Characterization Framework for Intelligent Sequential Decision Making](https://mlsanity.com/paper/2610.11696/) (2026)

## Abstract (arXiv)

Goals for reinforcement learning problems are typically defined through hand-specified rewards. To design such problems, developers of learning algorithms must inherently be aware of what the task goals are, yet we often require agents to discover them on their own without any supervision beyond these sparse rewards. While much of the power of reinforcement learning derives from the concept that agents can learn with little guidance, this requirement greatly burdens the training process. If we relax this one restriction and endow the agent with knowledge of the reward function, and in particular of the goal, we can leverage backwards induction to accelerate training. To achieve this, we propose training a model to learn to take imagined reversal steps from known goal states. Rather than training an agent exclusively to determine how to reach a goal while moving forwards in time, our approach travels backwards to jointly predict how we got there. We evaluate our work in Gridworld and Towers of Hanoi and empirically demonstrate that it yields better performance than standard DDQN.