# Learning Control from Raw Position Measurements

arXiv: https://arxiv.org/abs/2301.13183 · HTML: https://mlsanity.com/paper/2301.13183/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Model-Ensemble Trust-Region Policy Optimization](https://mlsanity.com/paper/1802.10592/) (2018)
- [Gaussian Processes for Data-Efficient Learning in Robotics and Control](https://mlsanity.com/paper/1502.02860/) (2015)
- [Black-Box Data-efficient Policy Search for Robotics](https://mlsanity.com/paper/1703.07261/) (2017)
- [Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application](https://mlsanity.com/paper/2101.12115/) (2021)
- [Derivative-free online learning of inverse dynamics models](https://mlsanity.com/paper/1809.05074/) (2018)
- [Semiparametrical Gaussian Processes Learning of Forward Dynamical Models for Navigating in a Circular Maze](https://mlsanity.com/paper/1809.04993/) (2018)
- [Model-Based Reinforcement Learning for Physical Systems Without Velocity and Acceleration Measurements](https://mlsanity.com/paper/2002.10621/) (2020)

## Built on by (most cited first, 3 of 3)

- [Learning control of underactuated double pendulum with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2409.05811/) (2024)
- [Learning global control of underactuated systems with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2504.06721/) (2025)
- [Accelerating Model-Based Reinforcement Learning using Non-Linear Trajectory Optimization](https://mlsanity.com/paper/2506.02767/) (2025)

## Abstract (arXiv)

We propose a Model-Based Reinforcement Learning (MBRL) algorithm named VF-MC-PILCO, specifically designed for application to mechanical systems where velocities cannot be directly measured. This circumstance, if not adequately considered, can compromise the success of MBRL approaches. To cope with this problem, we define a velocity-free state formulation which consists of the collection of past positions and inputs. Then, VF-MC-PILCO uses Gaussian Process Regression to model the dynamics of the velocity-free state and optimizes the control policy through a particle-based policy gradient approach. We compare VF-MC-PILCO with our previous MBRL algorithm, MC-PILCO4PMS, which handles the lack of direct velocity measurements by modeling the presence of velocity estimators. Results on both simulated (cart-pole and UR5 robot) and real mechanical systems (Furuta pendulum and a ball-and-plate rig) show that the two algorithms achieve similar results. Conveniently, VF-MC-PILCO does not require the design and implementation of state estimators, which can be a challenging and time-consuming activity to be performed by an expert user.