Learning to Linearize Under Uncertainty
Not on the map. Why?
Abstract (arXiv)
Training deep feature hierarchies to solve supervised learning tasks has achieved state of the art performance on many problems in computer vision. However, a principled way in which to train such hierarchies in the unsupervised setting has remained elusive. In this work we suggest a new architecture and loss for training deep feature hierarchies that linearize the transformations observed in unlabeled natural video sequences. This is done by training a generative model to predict video frames. We also address the problem of inherent uncertainty in prediction by introducing latent variables that are non-deterministic functions of the input into the network architecture.
Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Jun 2015.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 4; 2022: 5; 2023: 4; 2024: 6; 2025: 4; 2026: 2.
25 papers.
Sort:
By area:
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 120
- 2021Disentangling Hate in Online Memescited by 46
- 2026Temporal Straightening for Latent Planningcited by 38
- 2024Scaling 4D Representationscited by 30
- 2022Symmetry-Based Representations for Artificial and Biological General Intelligencecited by 26
- 2022Back to the Manifold: Recovering from Out-of-Distribution Statescited by 20
- 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
- 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 17
- 2021Localization with Sampling-Argmaxcited by 11
- 2022Unsupervised Multi-object Segmentation Using Attention and Soft-argmaxcited by 11
- 2025Chirality in Action: Time-Aware Video Representation Learning by Latent Straighteningcited by 11
- 2021PDWN: Pyramid Deformable Warping Network for Video Interpolationcited by 10
13 more
- 2024Learning predictable and robust neural representations by straightening image sequencescited by 9
- 2021Two-stage Rule-induction Visual Reasoning on RPMs with an Application to Video Predictioncited by 7
- 2024Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularizationcited by 7
- 2022Minimalistic Unsupervised Learning with the Sparse Manifold Transformcited by 6
- 2023A polar prediction model for learning to represent visual transformationscited by 6
- 2026GEOPHYS: The Geometry of Physical Plausibilitycited by 3
- 2023Long Range Object-Level Monocular Depth Estimation for UAVscited by 2
- 2023Prior based Sampling for Adaptive LiDARcited by 2
- 2023Brain-like representational straightening of natural movies in robust feedforward neural networkscited by 2
- 2025Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modelingcited by 2
- 2025SAM 2++: Tracking Anything at Any Granularitycited by 1
- 2024Video Prediction Models as General Visual Encoderscited by 0
- 2025Nonlinear Model Order Reduction of Dynamical Systems in Process Engineering: Review and Comparisoncited by 0
- 2026GEOPHYS: The Geometry of Physical Plausibility15 Jun 2026
- 2026Temporal Straightening for Latent Planning12 Mar 2026
- 2025SAM 2++: Tracking Anything at Any Granularity21 Oct 2025
- 2025Chirality in Action: Time-Aware Video Representation Learning by Latent Straightening10 Sep 2025
- 2025Nonlinear Model Order Reduction of Dynamical Systems in Process Engineering: Review and Comparison15 Jun 2025
- 2025Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling7 Apr 2025
- 2024Scaling 4D Representations19 Dec 2024
- 2024Learning predictable and robust neural representations by straightening image sequences4 Nov 2024
- 2024Video Prediction Models as General Visual Encoders25 May 2024
- 2024Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization22 Mar 2024
- 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference6 Mar 2024
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Point1 Feb 2024
13 more
- 2023Brain-like representational straightening of natural movies in robust feedforward neural networks26 Aug 2023
- 2023Prior based Sampling for Adaptive LiDAR14 Apr 2023
- 2023A polar prediction model for learning to represent visual transformations6 Mar 2023
- 2023Long Range Object-Level Monocular Depth Estimation for UAVs17 Feb 2023
- 2022Minimalistic Unsupervised Learning with the Sparse Manifold Transform30 Sep 2022
- 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Review27 Aug 2022
- 2022Back to the Manifold: Recovering from Out-of-Distribution States18 Jul 2022
- 2022Unsupervised Multi-object Segmentation Using Attention and Soft-argmax26 May 2022
- 2022Symmetry-Based Representations for Artificial and Biological General Intelligence17 Mar 2022
- 2021Two-stage Rule-induction Visual Reasoning on RPMs with an Application to Video Prediction24 Nov 2021
- 2021Localization with Sampling-Argmax17 Oct 2021
- 2021Disentangling Hate in Online Memes9 Aug 2021
- 2021PDWN: Pyramid Deformable Warping Network for Video Interpolation4 Apr 2021
- 2026Temporal Straightening for Latent Planningcited by 38
- 2026Temporal Straightening for Latent Planning12 Mar 2026
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 120
- 2021Disentangling Hate in Online Memescited by 46
- 2024Scaling 4D Representationscited by 30
- 2022Symmetry-Based Representations for Artificial and Biological General Intelligencecited by 26
- 2022Back to the Manifold: Recovering from Out-of-Distribution Statescited by 20
- 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
- 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 17
- 2021Localization with Sampling-Argmaxcited by 11
- 2022Unsupervised Multi-object Segmentation Using Attention and Soft-argmaxcited by 11
- 2025Chirality in Action: Time-Aware Video Representation Learning by Latent Straighteningcited by 11
- 2021PDWN: Pyramid Deformable Warping Network for Video Interpolationcited by 10
- 2024Learning predictable and robust neural representations by straightening image sequencescited by 9
12 most cited of 24 All 24 (JSON)
- 2026GEOPHYS: The Geometry of Physical Plausibility15 Jun 2026
- 2025SAM 2++: Tracking Anything at Any Granularity21 Oct 2025
- 2025Chirality in Action: Time-Aware Video Representation Learning by Latent Straightening10 Sep 2025
- 2025Nonlinear Model Order Reduction of Dynamical Systems in Process Engineering: Review and Comparison15 Jun 2025
- 2025Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modeling7 Apr 2025
- 2024Scaling 4D Representations19 Dec 2024
- 2024Learning predictable and robust neural representations by straightening image sequences4 Nov 2024
- 2024Video Prediction Models as General Visual Encoders25 May 2024
- 2024Pose-Aware Self-Supervised Learning with Viewpoint Trajectory Regularization22 Mar 2024
- 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference6 Mar 2024
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Point1 Feb 2024
- 2023Brain-like representational straightening of natural movies in robust feedforward neural networks26 Aug 2023
12 newest of 24 All 24 (JSON)