mlsanityData to 7 Oct 2026

Spatio-temporal Value Semantics-based Abstraction for Dense Deep Reinforcement Learning

arXiv 2405.15829 ↗First posted 24 May 2024cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Intelligent Cyber-Physical Systems (ICPS) represent a specialized form of Cyber-Physical System (CPS) that incorporates intelligent components, notably Convolutional Neural Networks (CNNs) and Deep Reinforcement Learning (DRL), to undertake multifaceted tasks encompassing perception, decision-making, and control. The utilization of DRL for decision-making facilitates dynamic interaction with the environment, generating control actions aimed at maximizing cumulative rewards. Nevertheless, the inherent uncertainty of the operational environment and the intricate nature of ICPS necessitate exploration within complex and dynamic state spaces during the learning phase. DRL confronts challenges in terms of efficiency, generalization capabilities, and data scarcity during decision-making process. In response to these challenges, we propose an innovative abstract modeling approach grounded in spatial-temporal value semantics, capturing the evolution in the distribution of semantic value across time and space. A semantics-based abstraction is introduced to construct an abstract Markov Decision Process (MDP) for the DRL learning process. Furthermore, optimization techniques for abstraction are delineated, aiming to refine the abstract model and mitigate semantic gaps between abstract and concrete states. The efficacy of the abstract modeling is assessed through the evaluation and analysis of the abstract MDP model using PRISM. A series of experiments are conducted, involving diverse scenarios such as lane-keeping, adaptive cruise control, and intersection crossroad assistance, to demonstrate the effectiveness of our abstracting approach.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 42 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

18 of 42 references.

References · 18

  1. 2016Enriching Word Vectors with Subword Informationcited by 1,578
  2. 2018Exploration by Random Network Distillationcited by 966
  3. 2016The Option-Critic Architecturecited by 433
  4. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  5. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  6. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  7. 2013Causal Inference in the Presence of Latent Variables and Selection Biascited by 165
  8. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  9. 2020MDP Homomorphic Networks: Group Symmetries in Reinforcement Learningcited by 133
  10. 2019Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learningcited by 114
  11. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  12. 2014Statistical-Computational Tradeoffs in Planted Problems and Submatrix Localization with a Growing Number of Clusters and Submatricescited by 29
6 more
  1. 2022A Theory of Abstraction in Reinforcement Learningcited by 26
  2. 2022An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobilitycited by 16
  3. 2022Phasic Self-Imitative Reduction for Sparse-Reward Goal-Conditioned Reinforcement Learningcited by 16
  4. 2022Constructing MDP Abstractions Using Data with Formal Guaranteescited by 7
  5. 2022Abstraction-Refinement for Hierarchical Probabilistic Modelscited by 5
  6. 2023Abstraction of Nondeterministic Situation Calculus Action Theories -- Extended Versioncited by 2

Built on by

Not cited yet.