A representational framework for learning and encoding structurally enriched trajectories in complex agent environments

arXiv 2503.13194 ↗First posted 17 Mar 2025cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The ability of artificial intelligence agents to make optimal decisions and generalise them to different domains and tasks is compromised in complex scenarios. One way to address this issue has focused on learning efficient representations of the world and on how the actions of agents affect them in state-action transitions. Whereas such representations are procedurally efficient, they lack structural richness. To address this problem, we propose to enhance the agent's ontology and extend the traditional conceptualisation of trajectories to provide a more nuanced view of task execution. Structurally Enriched Trajectories (SETs) extend the encoding of sequences of states and their transitions by incorporating hierarchical relations between objects, interactions, and affordances. SETs are built as multi-level graphs, providing a detailed representation of the agent dynamics and a transferable functional abstraction of the task. SETs are integrated into an architecture, Structurally Enriched Trajectory Learning and Encoding (SETLE), that employs a heterogeneous graph-based memory structure of multi-level relational dependencies essential for generalisation. We demonstrate that SETLE can support downstream tasks, enabling agents to recognise task relevant structural patterns across CREATE and MiniGrid environments. Finally, we integrate SETLE with reinforcement learning and show measurable improvements in downstream performance, including breakthrough success rates in complex, sparse-reward tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 60 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

25 of 60 references.

References · 25

2015Deep Residual Learning for Image Recognitioncited by 52,096
2023Segment Anythingcited by 8,234
2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
2017Neural Message Passing for Quantum Chemistrycited by 3,487
2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
2016The Option-Critic Architecturecited by 448
2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
2018Towards a Definition of Disentangled Representationscited by 296

13 more

2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
2021Self-supervised Heterogeneous Graph Neural Network with Co-contrastive Learningcited by 130
2018State Representation Learning for Control: An Overviewcited by 127
2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
2018Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddingscited by 52
2024Sharing Knowledge in Multi-Task Deep Reinforcement Learningcited by 44
2020Reinforcement Learning with Trajectory Feedbackcited by 40
2020Generalization to New Actions in Reinforcement Learningcited by 25
2021What Can I Do Here? Learning New Skills by Imagining Visual Affordancescited by 23
2021Outcome-Driven Reinforcement Learning via Variational Inferencecited by 17
2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
2024GTC: GNN-Transformer Co-contrastive Learning for Self-supervised Heterogeneous Graph Representationcited by 9
2021Sequential memory improves sample and memory efficiency in Episodic Controlcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way