# Using Memory-Based Learning to Solve Tasks with State-Action Constraints

arXiv: https://arxiv.org/abs/2303.04327 · HTML: https://mlsanity.com/paper/2303.04327/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 8 Mar 2023.

## Builds on (most cited first, 15 of 15)

- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [Go-Explore: a New Approach for Hard-Exploration Problems](https://mlsanity.com/paper/1901.10995/) (2019)
- [Rearrangement: A Challenge for Embodied AI](https://mlsanity.com/paper/2011.01975/) (2020)
- [Online Replanning in Belief Space for Partially Observable Task and Motion Problems](https://mlsanity.com/paper/1911.04577/) (2019)
- [Deep Affordance Foresight: Planning Through What Can Be Done in the Future](https://mlsanity.com/paper/2011.08424/) (2020)
- [Long-Horizon Multi-Robot Rearrangement Planning for Construction Assembly](https://mlsanity.com/paper/2106.02489/) (2021)
- [Large-Scale Retrieval for Reinforcement Learning](https://mlsanity.com/paper/2206.05314/) (2022)
- [Practical Kernel-Based Reinforcement Learning](https://mlsanity.com/paper/1407.5358/) (2014)
- [Dual Control for Approximate Bayesian Reinforcement Learning](https://mlsanity.com/paper/1510.03591/) (2015)
- [Fast High-Quality Tabletop Rearrangement in Bounded Workspace](https://mlsanity.com/paper/2110.12325/) (2021)
- [Relational Learning for Skill Preconditions](https://mlsanity.com/paper/2012.01693/) (2020)
- [Affordance-Based Mobile Robot Navigation Among Movable Obstacles](https://mlsanity.com/paper/2102.04918/) (2021)
- [Non-Prehensile Manipulation in Clutter with Human-In-The-Loop](https://mlsanity.com/paper/1904.03748/) (2019)
- [Learning Symbolic Operators: A Neurosymbolic Solution for Autonomous Disassembly of Electric Vehicle Battery](https://mlsanity.com/paper/2206.03027/) (2022)

## Built on by (most cited first, 2 of 2)

- [InCoRo: In-Context Learning for Robotics Control with Feedback Loops](https://mlsanity.com/paper/2402.05188/) (2024)
- [Probing an Embodied LLM: When Higher Observation Fidelity Hurts Problem Solving](https://mlsanity.com/paper/2605.20072/) (2026)

## Abstract (arXiv)

Tasks where the set of possible actions depend discontinuously on the state pose a significant challenge for current reinforcement learning algorithms. For example, a locked door must be first unlocked, and then the handle turned before the door can be opened. The sequential nature of these tasks makes obtaining final rewards difficult, and transferring information between task variants using continuous learned values such as weights rather than discrete symbols can be inefficient. Our key insight is that agents that act and think symbolically are often more effective in dealing with these tasks. We propose a memory-based learning approach that leverages the symbolic nature of constraints and temporal ordering of actions in these tasks to quickly acquire and transfer high-level information. We evaluate the performance of memory-based learning on both real and simulated tasks with approximately discontinuous constraints between states and actions, and show our method learns to solve these tasks an order of magnitude faster than both model-based and model-free deep reinforcement learning methods.