# Top-Down Guidance for Learning Object-Centric Representations

arXiv: https://arxiv.org/abs/2405.10598 · HTML: https://mlsanity.com/paper/2405.10598/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 41 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 25 of 25)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [Multi-Object Representation Learning with Iterative Variational Inference](https://mlsanity.com/paper/1903.00450/) (2019)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [Bridging the Gap to Real-World Object-Centric Learning](https://mlsanity.com/paper/2209.14860/) (2022)
- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models](https://mlsanity.com/paper/2210.05861/) (2022)
- [ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation](https://mlsanity.com/paper/2111.10265/) (2021)
- [Unsupervised Learning of Object Structure and Dynamics from Videos](https://mlsanity.com/paper/1906.07889/) (2019)
- [High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks](https://mlsanity.com/paper/1911.01655/) (2019)
- [FitVid: Overfitting in Pixel-Level Video Prediction](https://mlsanity.com/paper/2106.13195/) (2021)
- [Improving Object-centric Learning with Query Optimization](https://mlsanity.com/paper/2210.08990/) (2022)
- [Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames](https://mlsanity.com/paper/2302.04973/) (2023)
- [SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers](https://mlsanity.com/paper/2312.00648/) (2023)
- [Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation](https://mlsanity.com/paper/2207.00787/) (2022)
- [TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2112.01515/) (2021)
- [A Control-Centric Benchmark for Video Prediction](https://mlsanity.com/paper/2304.13723/) (2023)
- [Deep Predictive Coding Network for Object Recognition](https://mlsanity.com/paper/1802.04762/) (2018)
- [Top-down Visual Saliency Guided by Captions](https://mlsanity.com/paper/1612.07360/) (2016)
- [Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactions](https://mlsanity.com/paper/2302.11850/) (2023)

## Built on by (most cited first, 3 of 3)

- [Rethinking Object-Centric Representations for Video Dynamics Modeling](https://mlsanity.com/paper/2606.23436/) (2026)
- [Iterative Feature Exclusion Ranking for Deep Tabular Learning](https://mlsanity.com/paper/2412.16442/) (2024)
- [Object-Centric Conditioning for Visuomotor Flow Matching](https://mlsanity.com/paper/2609.24155/) (2026)

## Abstract (arXiv)

Humans' innate ability to decompose scenes into objects allows for efficient understanding, predicting, and planning. In light of this, Object-Centric Learning (OCL) attempts to endow networks with similar capabilities, learning to represent scenes with the composition of objects. However, existing OCL models only learn through reconstructing the input images, which does not assist the model in distinguishing objects, resulting in suboptimal object-centric representations. This flaw limits current object-centric models to relatively simple downstream tasks. To address this issue, we draw on humans' top-down vision pathway and propose Top-Down Guided Network (TDGNet), which includes a top-down pathway to improve object-centric representations. During training, the top-down pathway constructs guidance with high-level object-centric representations to optimize low-level grid features output by the backbone. While during inference, it refines object-centric representations by detecting and solving conflicts between low- and high-level features. We show that TDGNet outperforms current object-centric models on multiple datasets of varying complexity. In addition, we expand the downstream task scope of object-centric representations by applying TDGNet to the field of robotics, validating its effectiveness in downstream tasks including video prediction and visual planning.