# Reinforcement learning with timed constraints for robotics motion planning

arXiv: https://arxiv.org/abs/2601.00087 · HTML: https://mlsanity.com/paper/2601.00087/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 31 Dec 2025.

## Placed in

- Deep reinforcement learning methods / [Interpretable and logic-guided RL](https://mlsanity.com/deeprl/interpretable-and-logic-guided-rl/)

## Builds on (most cited first, 15 of 15)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic](https://mlsanity.com/paper/2102.12855/) (2021)
- [Transfer of Temporal Logic Formulas in Reinforcement Learning](https://mlsanity.com/paper/1909.04256/) (2019)
- [Multi-agent Motion Planning from Signal Temporal Logic Specifications](https://mlsanity.com/paper/2201.05247/) (2022)
- [Optimal Probabilistic Motion Planning with Potential Infeasible LTL Constraints](https://mlsanity.com/paper/2007.14325/) (2020)
- [Reinforcement Learning of Control Policy for Linear Temporal Logic Specifications Using Limit-Deterministic Generalized B"uchi Automata](https://mlsanity.com/paper/2001.04669/) (2020)
- [The Larger the Merrier? Efficient Large AI Model Inference in Wireless Edge Networks](https://mlsanity.com/paper/2505.09214/) (2025)
- [Safe motion planning with environment uncertainty](https://mlsanity.com/paper/2305.06004/) (2023)
- [Learning-based agricultural management in partially observable environments subject to climate variability](https://mlsanity.com/paper/2401.01273/) (2024)
- [Computational methods for stochastic control with metric interval temporal logic specifications](https://mlsanity.com/paper/1503.07193/) (2015)
- [Cooperative Planning for Coupled Multi-Agent Systems under Timed Temporal Specifications](https://mlsanity.com/paper/1603.05097/) (2016)
- [Control Synthesis for Multi-Agent Systems under Metric Interval Temporal Logic Specifications](https://mlsanity.com/paper/1703.02780/) (2017)
- [Temporally Robust Multi-Agent STL Motion Planning in Continuous Time](https://mlsanity.com/paper/2310.10585/) (2023)
- [Robust MITL planning under uncertain navigation times](https://mlsanity.com/paper/2403.03727/) (2024)

## Abstract (arXiv)

Robotic systems operating in dynamic and uncertain environments increasingly require planners that satisfy complex task sequences while adhering to strict temporal constraints. Metric Interval Temporal Logic (MITL) offers a formal and expressive framework for specifying such time-bounded requirements; however, integrating MITL with reinforcement learning (RL) remains challenging due to stochastic dynamics and partial observability. This paper presents a unified automata-based RL framework for synthesizing policies in both Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs) under MITL specifications. MITL formulas are translated into Timed Limit-Deterministic Generalized B"uchi Automata (Timed-LDGBA) and synchronized with the underlying decision process to construct product timed models suitable for Q-learning. A simple yet expressive reward structure enforces temporal correctness while allowing additional performance objectives. The approach is validated in three simulation studies: a 5 × 5 grid-world formulated as an MDP, a 10 × 10 grid-world formulated as a POMDP, and an office-like service-robot scenario. Results demonstrate that the proposed framework consistently learns policies that satisfy strict time-bounded requirements under stochastic transitions, scales to larger state spaces, and remains effective in partially observable environments, highlighting its potential for reliable robotic planning in time-critical and uncertain settings.