# Privileged observations enable rapid and reliable policy discovery directly in the physical world

arXiv: https://arxiv.org/abs/2512.08463 · HTML: https://mlsanity.com/paper/2512.08463/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 82 references that are in the map.
- First posted on arXiv 9 Dec 2025.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Asymmetric Actor Critic for Image-Based Robot Learning](https://mlsanity.com/paper/1710.06542/) (2017)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [Privileged Sensing Scaffolds Reinforcement Learning](https://mlsanity.com/paper/2405.14853/) (2024)
- [Deep Reinforcement Learning for Event-Triggered Control](https://mlsanity.com/paper/1809.05152/) (2018)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [Sample-Efficient Learning to Solve a Real-World Labyrinth Game Using Data-Augmented Model-Based Reinforcement Learning](https://mlsanity.com/paper/2312.09906/) (2023)
- [Active Learning for Control-Oriented Identification of Nonlinear Systems](https://mlsanity.com/paper/2404.09030/) (2024)
- [How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?](https://mlsanity.com/paper/2104.00827/) (2021)
- [Understanding Robust Control Theory Via Stick Balancing](https://mlsanity.com/paper/1606.03520/) (2016)
- [An Inversion-Based Learning Approach for Improving Impromptu Trajectory Tracking of Robots with Non-Minimum Phase Dynamics](https://mlsanity.com/paper/1709.04407/) (2017)
- [Deep Reinforcement Learning in Applied Control: Challenges, Analysis, and Insights](https://mlsanity.com/paper/2507.08196/) (2025)
- [Learning Tactile Insertion in the Real World](https://mlsanity.com/paper/2405.00383/) (2024)

## Built on by (most cited first, 3 of 3)

- [SynthPix: A lightspeed PIV image generator](https://mlsanity.com/paper/2512.09664/) (2025)
- [Particle Image Velocimetry Refinement via Consensus ADMM for Active Fluid Control](https://mlsanity.com/paper/2512.11695/) (2025)
- [Flow Gym: A framework for the development, benchmarking, training, and deployment of flow-field quantification methods](https://mlsanity.com/paper/2512.20642/) (2025)

## Abstract (arXiv)

We study how privileged information about a physical system affects the discovery of high-performing policies when training a reinforcement learning agent directly in the physical world. We let the agent control a cylinder in a tabletop water channel to maximize or minimize drag. The flow is chaotic and difficult to model or simulate accurately and good strategies are not obvious beforehand. Decades-old experimental studies provide recipes for simple, high-performance, periodic open-loop policies that increase drag by 26.6% +/- 0.7% and reduce it by 29.7% +/- 1.3%. With dense observations of the cylinder wake, the agent learns within tens of minutes policies that increase drag by 25.5% +/- 0.9% and reduce it by 32.4% +/- 1.6%. We record action trajectories during online policy execution and replay them open loop as fixed sequences; these replays increase drag by 23.2% +/- 2.2% and reduce it by 32.1% +/- 3.0%. However, when we withhold flow observations during training, the agent still learns to decrease drag by 31.4% +/- 2.1%, but no run learns to increase it (1.8% +/- 4.3% drag increase). Our physical experiments demonstrate an extreme case: privileged observations can be decisive for policy discovery even when the resulting actions can be replayed in open loop.