Performance-Driven Environment Abstraction with Multi-Timescale Learning
On the map: Deep reinforcement learning methods → Deep RL algorithms and representations
Abstract (arXiv)
We study performance-driven environment abstraction for decision-making in large Markov decision processes. Rather than preserving geometric or topological structure, we seek abstractions that directly optimize decision quality. We model abstraction as a controlled approximation obtained by aggregating the state space and enforcing a shared action distribution within each aggregated state. For a fixed partition, we establish a performance guarantee that separates value-function approximation error from the loss introduced by action sharing. Guided by this analysis, we develop a multi-timescale reinforcement learning framework that jointly adapts the policy and a tree-structured environment abstraction. The resulting algorithm refines and coarsens regions of the state space based on Q-value discrepancies, balancing performance against abstraction size and complexity. Empirical results demonstrate substantial state compression, improved sample efficiency, and faster replanning compared to actor-critic baselines.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 22 references that are in the map. First posted on arXiv 16 Jun 2026.
Builds on
6 of 22 references.
References · 6
- 2017Eigenoption Discovery through the Deep Successor Representationcited by 86
- 2023Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systemscited by 60
- 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 52
- 2022Intelligent problem-solving as integrated hierarchical reinforcement learningcited by 21
- 2022Hierarchical Decompositions of Stochastic Pursuit-Evasion Gamescited by 2
- 2022Information-theoretic Abstraction of Semantic Octree Models for Integrated Perception and Planningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning with Expert Abstractions for Efficient Multi-Task Continuous Control
- 2025Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learning
- 2026Low-Complexity Policy Tessellations in Structured Markov Decision Processes
- 2025Context-Sensitive Abstractions for Reinforcement Learning with Parameterized Actions
- 2025SPOT: Scalable Policy Optimization with Trees for Markov Decision Processes
- 2024Reconciling Spatial and Temporal Abstractions for Goal Representation
- 2026Policy Gradient Methods for Non-Markovian Reinforcement Learning
- 2026Q-Shaped Options for Hierarchical Reinforcement Learning