Live LTL Progress Tracking: Towards Task-Based Exploration
On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL
Abstract (arXiv)
Motivated by the challenge presented by non-Markovian objectives in reinforcement learning (RL), we present a novel framework to track and represent the progress of autonomous agents through complex, multi-stage tasks. Given a specification in finite linear temporal logic (LTL), the framework establishes a 'tracking vector' which updates at each time step in a trajectory rollout. The values of the vector represent the status of the specification as the trajectory develops, assigning true, false, or 'open' labels (where 'open' is used for indeterminate cases). Applied to an LTL formula tree, the tracking vector can be used to encode detailed information about how a task is executed over a trajectory, providing a potential tool for new performance metrics, diverse exploration, and reward shaping. In this paper, we formally present the framework and algorithm, collectively named Live LTL Progress Tracking, give a simple working example, and demonstrate avenues for its integration into RL models. Future work will apply the framework to problems such as task-space exploration and diverse solution-finding in RL.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 42 references that are in the map. First posted on arXiv 18 Apr 2026.
Builds on
19 of 42 references.
Closest ancestors · 5
Same direction, newest first
- 2024Adaptive Reward Design for Reinforcement Learning14 Dec
- 2024Beyond Simple Sum of Delayed Rewards: Non-Markovian Reward Modeling for Reinforcement Learning26 Oct
- 2024Reinforcement Learning with LTL and ω-Regular Objectives via Optimality-Preserving Translation to Average Rewards16 Oct
- 2024Directed Exploration in Reinforcement Learning from Linear Temporal Logic18 Aug
- 2024LTL-Constrained Policy Optimization with Cycle Experience Replay17 Apr
Other references · 14
- 2015LSTM: A Search Space Odysseycited by 316
- 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
- 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
- 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 74
- 2014Probably Approximately Correct MDP Learning and Control With Temporal Logic Constraintscited by 54
- 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 38
- 2021Learning Long-Term Reward Redistribution via Randomized Return Decompositioncited by 33
- 2019Modular Deep Reinforcement Learning with Temporal Logic Specificationscited by 25
- 2017The Effects of Memory Replay in Reinforcement Learningcited by 23
- 2017A Policy Search Method For Temporal Logic Specified Reinforcement Learning Taskscited by 20
- 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
- 2022Accelerated Reinforcement Learning for Temporal Logic Control Objectivescited by 9
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Sample-Efficient Reinforcement Learning with Temporal Logic Objectives: Leveraging the Task Specification to Guide Exploration
- 2025Accelerated Learning with Linear Temporal Logic using Differentiable Simulation
- 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RL
- 2025SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attribution
- 2026Semantically Labelled Automata for Multi-Task Reinforcement Learning with LTL Instructions
- 2024Comp-LTL: Temporal Logic Planning via Zero-Shot Policy Composition
- 2024Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agents
- 2026Explore More, Drift Less: Outcome-Only Reinforcement Learning Can Suffice for Long-Horizon Interactive Agents