# Language-Mediated, Object-Centric Representation Learning

arXiv: https://arxiv.org/abs/2012.15814 · HTML: https://mlsanity.com/paper/2012.15814/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2020.

## Built on by (most cited first, 11 of 11)

- [FILM: Following Instructions in Language with Modular Methods](https://mlsanity.com/paper/2110.07342/) (2021)
- [PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning](https://mlsanity.com/paper/2112.05136/) (2021)
- [3D Concept Grounding on Neural Fields](https://mlsanity.com/paper/2207.06403/) (2022)
- [CTRL-O: Language-Controllable Object-Centric Visual Representation Learning](https://mlsanity.com/paper/2503.21747/) (2025)
- [Voxel-informed Language Grounding](https://mlsanity.com/paper/2205.09710/) (2022)
- [Embodied Concept Learner: Self-supervised Learning of Concepts and Mapping through Instruction Following](https://mlsanity.com/paper/2304.03767/) (2023)
- [Attentive Illumination Decomposition Model for Multi-Illuminant White Balancing](https://mlsanity.com/paper/2402.18277/) (2024)
- [Learning the meanings of function words from grounded language using a visual question answering model](https://mlsanity.com/paper/2308.08628/) (2023)
- [Acquiring Linguistic Knowledge from Multimodal Input](https://mlsanity.com/paper/2402.17936/) (2024)
- [Learning Planning Abstractions from Language](https://mlsanity.com/paper/2405.03864/) (2024)
- [HSA: Hierarchical Slot Attention for Multi-granularity Scene-Decomposition](https://mlsanity.com/paper/2607.08249/) (2026)

## Abstract (arXiv)

We present Language-mediated, Object-centric Representation Learning (LORL), a paradigm for learning disentangled, object-centric scene representations from vision and language. LORL builds upon recent advances in unsupervised object discovery and segmentation, notably MONet and Slot Attention. While these algorithms learn an object-centric representation just by reconstructing the input image, LORL enables them to further learn to associate the learned representations to concepts, i.e., words for object categories, properties, and spatial relationships, from language input. These object-centric concepts derived from language facilitate the learning of object-centric representations. LORL can be integrated with various unsupervised object discovery algorithms that are language-agnostic. Experiments show that the integration of LORL consistently improves the performance of unsupervised object discovery methods on two datasets via the help of language. We also show that concepts learned by LORL, in conjunction with object discovery methods, aid downstream tasks such as referring expression comprehension.