# Deep Object-Centric Representations for Generalizable Robot Learning

arXiv: https://arxiv.org/abs/1708.04225 · HTML: https://mlsanity.com/paper/1708.04225/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Aug 2017.

## Built on by (most cited first, 30 of 50)

- [VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors](https://mlsanity.com/paper/2210.11339/) (2022)
- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [Learning Generalizable Manipulation Policies with Object-Centric 3D Representations](https://mlsanity.com/paper/2310.14386/) (2023)
- [Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstruction](https://mlsanity.com/paper/2409.18121/) (2024)
- [Vision-based Manipulation from Single Human Video with Open-World Object Graphs](https://mlsanity.com/paper/2405.20321/) (2024)
- [Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstraction](https://mlsanity.com/paper/2306.01439/) (2023)
- [ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models](https://mlsanity.com/paper/2506.16211/) (2025)
- [Learning Manipulation by Predicting Interaction](https://mlsanity.com/paper/2406.00439/) (2024)
- [AffordDP: Generalizable Diffusion Policy with Transferable Affordance](https://mlsanity.com/paper/2412.03142/) (2024)
- [Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations](https://mlsanity.com/paper/2408.04380/) (2024)
- [Compositional Scene Representation Learning via Reconstruction: A Survey](https://mlsanity.com/paper/2202.07135/) (2022)
- [Look Wide and Interpret Twice: Improving Performance on Interactive Instruction-following Tasks](https://mlsanity.com/paper/2106.00596/) (2021)
- [Is an Object-Centric Video Representation Beneficial for Transfer?](https://mlsanity.com/paper/2207.10075/) (2022)
- [Structured World Belief for Reinforcement Learning in POMDP](https://mlsanity.com/paper/2107.08577/) (2021)
- [Visuomotor Control in Multi-Object Scenes Using Object-Aware Representations](https://mlsanity.com/paper/2205.06333/) (2022)
- [Slot-VAE: Object-Centric Scene Generation with Slot Attention](https://mlsanity.com/paper/2306.06997/) (2023)
- [Keypoint Abstraction using Large Models for Object-Relative Imitation Learning](https://mlsanity.com/paper/2410.23254/) (2024)
- [Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspective](https://mlsanity.com/paper/2511.11478/) (2025)
- [Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2108.03319/) (2021)
- [Self-supervised Reinforcement Learning with Independently Controllable Subgoals](https://mlsanity.com/paper/2109.04150/) (2021)
- [Composing Pre-Trained Object-Centric Representations for Robotics From "What" and "Where" Foundation Models](https://mlsanity.com/paper/2404.13474/) (2024)
- [P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies](https://mlsanity.com/paper/2412.06784/) (2024)
- [Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations](https://mlsanity.com/paper/2203.02107/) (2022)
- [Language-Guided Object-Centric Diffusion Policy for Generalizable and Collision-Aware Robotic Manipulation](https://mlsanity.com/paper/2407.00451/) (2024)
- [Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learning](https://mlsanity.com/paper/2501.16443/) (2025)
- [Subtask-Aware Visual Reward Learning from Segmented Demonstrations](https://mlsanity.com/paper/2502.20630/) (2025)
- [SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulation](https://mlsanity.com/paper/2511.06754/) (2025)
- [Out-of-Distribution Recovery with Object-Centric Keypoint Inverse Policy for Visuomotor Imitation Learning](https://mlsanity.com/paper/2411.03294/) (2024)
- [Agent-Centric Representations for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2104.09402/) (2021)
- [GRID: A Platform for General Robot Intelligence Development](https://mlsanity.com/paper/2310.00887/) (2023)

## Abstract (arXiv)

Robotic manipulation in complex open-world scenarios requires both reliable physical manipulation skills and effective and generalizable perception. In this paper, we propose a method where general purpose pretrained visual models serve as an object-centric prior for the perception system of a learned policy. We devise an object-level attentional mechanism that can be used to determine relevant objects from a few trajectories or demonstrations, and then immediately incorporate those objects into a learned policy. A task-independent meta-attention locates possible objects in the scene, and a task-specific attention identifies which objects are predictive of the trajectories. The scope of the task-specific attention is easily adjusted by showing demonstrations with distractor objects or with diverse relevant objects. Our results indicate that this approach exhibits good generalization across object instances using very few samples, and can be used to learn a variety of manipulation tasks using reinforcement learning.