# Learning image representations tied to ego-motion

arXiv: https://arxiv.org/abs/1505.02206 · HTML: https://mlsanity.com/paper/1505.02206/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2015.

## Built on by (most cited first, 30 of 42)

- [A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning](https://mlsanity.com/paper/2104.14558/) (2021)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning](https://mlsanity.com/paper/2603.14482/) (2026)
- [Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspective](https://mlsanity.com/paper/2103.17263/) (2021)
- [Equivariant Contrastive Learning](https://mlsanity.com/paper/2111.00899/) (2021)
- [Learning Task Informed Abstractions](https://mlsanity.com/paper/2106.15612/) (2021)
- [SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations](https://mlsanity.com/paper/2112.04680/) (2021)
- [SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency](https://mlsanity.com/paper/2112.01001/) (2021)
- [Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video](https://mlsanity.com/paper/2310.08584/) (2023)
- [Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2204.13060/) (2022)
- [Contrasting Contrastive Self-Supervised Representation Learning Pipelines](https://mlsanity.com/paper/2103.14005/) (2021)
- [Video Autoencoder: self-supervised disentanglement of static 3D structure and motion](https://mlsanity.com/paper/2110.02951/) (2021)
- [EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understanding](https://mlsanity.com/paper/2301.02217/) (2023)
- [Unsupervised Visual Representation Learning by Synchronous Momentum Grouping](https://mlsanity.com/paper/2207.06167/) (2022)
- [EgoEnv: Human-centric environment representations from egocentric video](https://mlsanity.com/paper/2207.11365/) (2022)
- [EquiMod: An Equivariance Module to Improve Self-Supervised Learning](https://mlsanity.com/paper/2211.01244/) (2022)
- [Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency](https://mlsanity.com/paper/2105.06463/) (2021)
- [Temporal Coherent Test-Time Optimization for Robust Video Classification](https://mlsanity.com/paper/2302.14309/) (2023)
- [Learning Temporal Dynamics from Cycles in Narrated Video](https://mlsanity.com/paper/2101.02337/) (2021)
- [Self-Supervised Video Similarity Learning](https://mlsanity.com/paper/2304.03378/) (2023)
- [General Intelligence Requires Reward-based Pretraining](https://mlsanity.com/paper/2502.19402/) (2025)
- [DRO: Deep Recurrent Optimizer for Video to Depth](https://mlsanity.com/paper/2103.13201/) (2021)
- [Self-supervised visual learning from interactions with objects](https://mlsanity.com/paper/2407.06704/) (2024)
- [Steerable Equivariant Representation Learning](https://mlsanity.com/paper/2302.11349/) (2023)
- [MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning](https://mlsanity.com/paper/2506.08694/) (2025)
- [Learning Neural Representation of Camera Pose with Matrix Representation of Pose Shift via View Synthesis](https://mlsanity.com/paper/2104.01508/) (2021)
- [ALP: Action-Aware Embodied Learning for Perception](https://mlsanity.com/paper/2306.10190/) (2023)
- [Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations](https://mlsanity.com/paper/2208.02680/) (2022)
- [Extraneousness-Aware Imitation Learning](https://mlsanity.com/paper/2210.01379/) (2022)

## Abstract (arXiv)

Understanding how images of objects and scenes behave in response to specific ego-motions is a crucial aspect of proper visual development, yet existing visual learning methods are conspicuously disconnected from the physical source of their images. We propose to exploit proprioceptive motor signals to provide unsupervised regularization in convolutional neural networks to learn visual representations from egocentric video. Specifically, we enforce that our learned features exhibit equivariance i.e. they respond predictably to transformations associated with distinct ego-motions. With three datasets, we show that our unsupervised feature learning approach significantly outperforms previous approaches on visual recognition and next-best-view prediction tasks. In the most challenging test, we show that features learned from video captured on an autonomous driving platform improve large-scale scene recognition in static images from a disjoint domain.