# Online Object Representations with Contrastive Learning

arXiv: https://arxiv.org/abs/1906.04312 · HTML: https://mlsanity.com/paper/1906.04312/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 13 of 13)

- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [Language Conditioned Imitation Learning over Unstructured Data](https://mlsanity.com/paper/2005.07648/) (2020)
- [Human-to-Robot Imitation in the Wild](https://mlsanity.com/paper/2207.09450/) (2022)
- [Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos](https://mlsanity.com/paper/2103.16817/) (2021)
- [Humanoid Policy ~ Human Policy](https://mlsanity.com/paper/2503.13441/) (2025)
- [Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers](https://mlsanity.com/paper/2403.12943/) (2024)
- [DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control](https://mlsanity.com/paper/2409.12192/) (2024)
- [Object-aware Contrastive Learning for Debiased Scene Representation](https://mlsanity.com/paper/2108.00049/) (2021)
- [Interactron: Embodied Adaptive Object Detection](https://mlsanity.com/paper/2202.00660/) (2022)
- [Learning Video-Conditioned Policies for Unseen Manipulation Tasks](https://mlsanity.com/paper/2305.06289/) (2023)
- [OSSID: Online Self-Supervised Instance Detection by (and for) Pose Estimation](https://mlsanity.com/paper/2201.07309/) (2022)
- [Robust and Controllable Object-Centric Learning through Energy-based Models](https://mlsanity.com/paper/2210.05519/) (2022)
- [Robots Autonomously Detecting People: A Multimodal Deep Contrastive Learning Method Robust to Intraclass Variations](https://mlsanity.com/paper/2203.00187/) (2022)

## Abstract (arXiv)

We propose a self-supervised approach for learning representations of objects from monocular videos and demonstrate it is particularly useful in situated settings such as robotics. The main contributions of this paper are: 1) a self-supervising objective trained with contrastive learning that can discover and disentangle object attributes from video without using any labels; 2) we leverage object self-supervision for online adaptation: the longer our online model looks at objects in a video, the lower the object identification error, while the offline baseline remains with a large fixed error; 3) to explore the possibilities of a system entirely free of human supervision, we let a robot collect its own data, train on this data with our self-supervise scheme, and then show the robot can point to objects similar to the one presented in front of it, demonstrating generalization of object attributes. An interesting and perhaps surprising finding of this approach is that given a limited set of objects, object correspondences will naturally emerge when using contrastive learning without requiring explicit positive pairs. Videos illustrating online object adaptation and robotic pointing are available at: https://online-objects.github.io/.