mlsanityData to 14 Oct 2026

Synthesizing Evolving Symbolic Representations for Autonomous Systems

arXiv 2409.11756 ↗First posted 18 Sep 2024cs.AI cs.SCcited by 0

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Recently, AI systems have made remarkable progress in various tasks. Deep Reinforcement Learning(DRL) is an effective tool for agents to learn policies in low-level state spaces to solve highly complex tasks. Researchers have introduced Intrinsic Motivation(IM) to the RL mechanism, which simulates the agent's curiosity, encouraging agents to explore interesting areas of the environment. This new feature has proved vital in enabling agents to learn policies without being given specific goals. However, even though DRL intelligence emerges through a sub-symbolic model, there is still a need for a sort of abstraction to understand the knowledge collected by the agent. To this end, the classical planning formalism has been used in recent research to explicitly represent the knowledge an autonomous agent acquires and effectively reach extrinsic goals. Despite classical planning usually presents limited expressive capabilities, PPDDL demonstrated usefulness in reviewing the knowledge gathered by an autonomous system, making explicit causal correlations, and can be exploited to find a plan to reach any state the agent faces during its experience. This work presents a new architecture implementing an open-ended learning system able to synthesize from scratch its experience into a PPDDL representation and update it over time. Without a predefined set of goals and tasks, the system integrates intrinsic motivations to explore the environment in a self-directed way, exploiting the high-level knowledge acquired during its experience. The system explores the environment and iteratively: (a) discover options, (b) explore the environment using options, (c) abstract the knowledge collected and (d) plan. This paper proposes an alternative approach to implementing open-ended learning architectures exploiting low-level and high-level representations to extend its knowledge in a virtuous loop.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 83 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

18 of 83 references.

References · 18

  1. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  2. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 639
  3. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 213
  4. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
  5. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  6. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 110
  7. 2017Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learningcited by 83
  8. 2018CURIOUS: Intrinsically Motivated Modular Multi-Goal Reinforcement Learningcited by 79
  9. 2017Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundarycited by 58
  10. 2018Reinforcement Learning for UAV Attitude Controlcited by 48
  11. 2016Towards Deep Symbolic Reinforcement Learningcited by 42
  12. 2021Interesting Object, Curious Agent: Learning Task-Agnostic Explorationcited by 38
6 more
  1. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
  2. 2019Unsupervised Grounding of Plannable First-Order Logic Representation from Imagescited by 24
  3. 2023A Definition of Open-Ended Learning Problems for Goal-Conditioned Agentscited by 18
  4. 2022Combining Fast and Slow Thinking for Human-like and Efficient Navigation in Constrained Environmentscited by 5
  5. 2019Learning High-Level Planning Symbols from Intrinsically Motivated Experiencecited by 1
  6. 2022Option Discovery for Autonomous Generation of Symbolic Knowledgecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Review of Deep Reinforcement Learning for Autonomous Driving
  2. 2024Decision Theory-Guided Deep Reinforcement Learning for Fast Learning
  3. 2025Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environments
  4. 2023Towards Safe Autonomous Driving Policies using a Neuro-Symbolic Deep Reinforcement Learning Approach
  5. 2025The Emergence of Deep Reinforcement Learning for Path Planning
  6. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning
  7. 2026Analyzing Symbolic Properties for DRL Agents in Systems and Networking
  8. 2024Deep Reinforcement Learning for Mobile Robot Path Planning