Recursive Backwards Q-Learning in Deterministic Environments
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
Reinforcement learning is a popular method of finding optimal solutions to complex problems. Algorithms like Q-learning excel at learning to solve stochastic problems without a model of their environment. However, they take longer to solve deterministic problems than is necessary. Q-learning can be improved to better solve deterministic problems by introducing such a model-based approach. This paper introduces the recursive backwards Q-learning (RBQL) agent, which explores and builds a model of the environment. After reaching a terminal state, it recursively propagates its value backwards through this model. This lets each state be evaluated to its optimal value without a lengthy learning process. In the example of finding the shortest path through a maze, this agent greatly outperforms a regular Q-learning agent.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 0 of its 8 references that are in the map. First posted on arXiv 24 Apr 2024.
Builds on
0 of 8 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ReversedQ: Opportunities for Faster Q-Learning in Episodic Online Reinforcement Learning
- 2024Towards Adapting Reinforcement Learning Agents to New Tasks: Insights from Q-Values
- 2026Reversal Q-Learning
- 2026Recursive Agent Optimization
- 2024From Two-Dimensional to Three-Dimensional Environment with Q-Learning: Modeling Autonomous Navigation with Reinforcement Learning and no Libraries
- 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review
- 2025Convergence of regularized agent-state-based Q-learning in POMDPs
- 2023Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization