mlsanityData to 7 Oct 2026

Expanding Spatial and Temporal Context for Robotic Imitation Learning With Scene Graphs

arXiv 2606.01072 ↗First posted 31 May 2026cs.RO cs.CVcited by 0

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Imitation learning enables robots to learn how to execute tasks via observation. However, real-world environments like homes and offices are often severely partially observed due to their large spatial scales. In addition, many tasks involve executing a series of subtasks requiring autonomous robots to reason over extended time horizons. To address these challenges, we propose using scene graphs as an explicit and structured memory mechanism in imitation learning. By maintaining a dynamic scene graph that captures object-centric relationships and their evolution over time, our method allows the agent to retain relevant historical context during task execution to efficiently reason over incrementally accrued scene information. Our experiments on simulated mobile manipulation and real-world tabletop manipulation demonstrate that our approach substantially improves policy performance, particularly in settings that demand long-term reasoning and robust generalization under partial observability.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 56 of its 65 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

56 of 65 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training1 Sep
  2. 2025What Matters in Learning from Large-Scale Datasets for Robot Manipulation16 Jun
  3. 2024Enabling Stateful Behaviors for Diffusion-based Policy Learning18 Apr
  4. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representations22 Oct
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr

Other references · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Segment Anythingcited by 8,158
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  8. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  9. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  10. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  11. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  12. 2021Implicit Behavioral Cloningcited by 442
38 more
  1. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  2. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  3. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  4. 2023Recognize Anything: A Strong Image Tagging Modelcited by 305
  5. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
  6. 2018Graph R-CNN for Scene Graph Generationcited by 296
  7. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  8. 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 280
  9. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  10. 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 165
  11. 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 159
  12. 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
  13. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  14. 2024Clio: Real-time Task-Driven Open-Set 3D Scene Graphscited by 81
  15. 2023Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systemscited by 60
  16. 2021Reinforcement Learning, Bit by Bitcited by 59
  17. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 50
  18. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  19. 2019Graph-Structured Visual Imitationcited by 43
  20. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 43
  21. 2021Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent Statescited by 42
  22. 2024Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulationcited by 38
  23. 2023Context-Aware Entity Grounding with Open-Vocabulary 3D Scene Graphscited by 33
  24. 2025Learning Long-Context Diffusion Policies via Past-Token Predictioncited by 28
  25. 2018Visual Graphs from Motion (VGfM): Scene understanding with object geometry reasoningcited by 27
  26. 2023GRID: Scene-Graph-based Instruction-driven Robotic Task Planningcited by 27
  27. 2019On Value Functions and the Agent-Environment Boundarycited by 19
  28. 2024Composing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Modelscited by 16
  29. 2025Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learningcited by 14
  30. 2024Human-inspired Perspectives: A Survey on AI Long-term Memorycited by 12
  31. 2022Visual representations in the human brain are aligned with large language modelscited by 11
  32. 2024Agent-state based policies in POMDPs: Beyond belief-state MDPscited by 9
  33. 2025ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Taskscited by 9
  34. 2025ASHiTA: Automatic Scene-grounded HIerarchical Task Analysiscited by 8
  35. 2024Task-Oriented Hierarchical Object Decomposition for Visuomotor Controlcited by 6
  36. 2025Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action modelscited by 6
  37. 2024Zero-shot Object-Centric Instruction Following: Integrating Foundation Models with Traditional Navigationcited by 4
  38. 2025HELIOS: Hierarchical Exploration for Language-Grounded Interaction in Open Scenescited by 2

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training and Evaluating Diffusion Policies with Long Context Lengths
  2. 2024Instant Policy: In-Context Imitation Learning via Graph Diffusion
  3. 2023Few-Shot In-Context Imitation Learning via Implicit Graph Alignment
  4. 2026BPP: Long-Context Robot Imitation Learning by Focusing on Key History Frames
  5. 2025Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditions
  6. 2025Compose by Focus: Scene Graph-based Atomic Skills
  7. 2026ICLR: In-Context Imitation Learning with Visual Reasoning
  8. 2026GSR: Learning Structured Reasoning for Embodied Manipulation ↗