# PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations

arXiv: https://arxiv.org/abs/1705.09805 · HTML: https://mlsanity.com/paper/1705.09805/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2017.

## Built on by (most cited first, 16 of 16)

- [Reinforcement Learning with Neural Radiance Fields](https://mlsanity.com/paper/2206.01634/) (2022)
- [Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2204.13060/) (2022)
- [Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation](https://mlsanity.com/paper/2304.06600/) (2023)
- [Unsupervised Representation Learning in Deep Reinforcement Learning: A Review](https://mlsanity.com/paper/2208.14226/) (2022)
- [A Survey of State Representation Learning for Deep Reinforcement Learning](https://mlsanity.com/paper/2506.17518/) (2025)
- [Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Vision](https://mlsanity.com/paper/2210.15206/) (2022)
- [Reinforcement Learning with Generalizable Gaussian Splatting](https://mlsanity.com/paper/2404.07950/) (2024)
- [Integrating Contrastive Learning with Dynamic Models for Reinforcement Learning from Images](https://mlsanity.com/paper/2203.01810/) (2022)
- [Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spaces](https://mlsanity.com/paper/2107.01667/) (2021)
- [State Representation Learning from Demonstration](https://mlsanity.com/paper/1910.01738/) (2019)
- [MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learning](https://mlsanity.com/paper/2409.02714/) (2024)
- [Low-Dimensional State and Action Representation Learning with MDP Homomorphism Metrics](https://mlsanity.com/paper/2107.01677/) (2021)
- [Distilling Governing Laws and Source Input for Dynamical Systems from Videos](https://mlsanity.com/paper/2205.01314/) (2022)
- [Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos](https://mlsanity.com/paper/2106.04776/) (2021)
- [Learn Proportional Derivative Controllable Latent Space from Pixels](https://mlsanity.com/paper/2110.08239/) (2021)
- [Locally Constrained Representations in Reinforcement Learning](https://mlsanity.com/paper/2209.09441/) (2022)

## Abstract (arXiv)

We propose position-velocity encoders (PVEs) which learn---without supervision---to encode images to positions and velocities of task-relevant objects. PVEs encode a single image into a low-dimensional position state and compute the velocity state from finite differences in position. In contrast to autoencoders, position-velocity encoders are not trained by image reconstruction, but by making the position-velocity representation consistent with priors about interacting with the physical world. We applied PVEs to several simulated control tasks from pixels and achieved promising preliminary results.