mlsanityData to 14 Oct 2026

Stratifying Reinforcement Learning with Signal Temporal Logic

arXiv 2604.04923 ↗First posted 6 Apr 2026cs.LG cs.LO cs.SYcited by 0

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

In this paper, we develop a stratification-based semantics for Signal Temporal Logic (STL) in which each atomic predicate is interpreted as a membership test in a stratified space. This perspective reveals a novel correspondence principle between stratification theory and STL, showing that most STL formulas can be viewed as inducing a stratification of space-time. The significance of this interpretation is twofold. First, it offers a fresh theoretical framework for analyzing the structure of the embedding space generated by deep reinforcement learning (DRL) and relates it to the geometry of the ambient decision space. Second, it provides a principled framework that both enables the reuse of existing high-dimensional analysis tools and motivates the creation of novel computational techniques. To ground the theory, we (1) illustrate the role of stratification theory in Minigrid games and (2) apply numerical techniques to the latent embeddings of a DRL agent playing such a game where the robustness of STL formulas is used as the reward. In the process, we propose computationally efficient signatures that, based on preliminary evidence, appear promising for uncovering the stratification structure of such embedding spaces.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 25 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

12 of 25 references.

References · 12

  1. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  2. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  3. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 250
  4. 2023The geometry of hidden representations of large transformer modelscited by 112
  5. 2022Verifying the Union of Manifolds Hypothesis for Image Datacited by 64
  6. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  7. 2019Estimating the effective dimension of large biological datasets using Fisher separability analysiscited by 19
  8. 2023Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agentscited by 16
  9. 2025Token embeddings violate the manifold hypothesiscited by 16
  10. 2026Topology and Geometry of the Learning Space of ReLU Networks: Connectivity and Singularitiescited by 6
  11. 2023HADES: Fast Singularity Detection with Local Measure Comparisoncited by 4
  12. 2025Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transformcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025STACHE: Local Black-Box Explanations for Reinforcement Learning Policies
  2. 2025TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks
  3. 2024STLGame: Signal Temporal Logic Games in Adversarial Multi-Agent Systems
  4. 2026State Trace Rationale As Auxiliary Task in Reinforcement Learning
  5. 2026Reward Machines for Signal Temporal Logic
  6. 2025Stratified GRPO: Handling Structural Heterogeneity in Reinforcement Learning of LLM Search Agents
  7. 2026Shielded Reinforcement Learning Under Dynamic Temporal Logic Constraints
  8. 2026Can RL Teach Long-Horizon Reasoning to LLMs? Expressiveness Is Key