Privileged observations enable rapid and reliable policy discovery directly in the physical world
Not on the map. Why?
Abstract (arXiv)
We study how privileged information about a physical system affects the discovery of high-performing policies when training a reinforcement learning agent directly in the physical world. We let the agent control a cylinder in a tabletop water channel to maximize or minimize drag. The flow is chaotic and difficult to model or simulate accurately and good strategies are not obvious beforehand. Decades-old experimental studies provide recipes for simple, high-performance, periodic open-loop policies that increase drag by 26.6% +/- 0.7% and reduce it by 29.7% +/- 1.3%. With dense observations of the cylinder wake, the agent learns within tens of minutes policies that increase drag by 25.5% +/- 0.9% and reduce it by 32.4% +/- 1.6%. We record action trajectories during online policy execution and replay them open loop as fixed sequences; these replays increase drag by 23.2% +/- 2.2% and reduce it by 32.1% +/- 3.0%. However, when we withhold flow observations during training, the agent still learns to decrease drag by 31.4% +/- 2.1%, but no run learns to increase it (1.8% +/- 4.3% drag increase). Our physical experiments demonstrate an extreme case: privileged observations can be decisive for policy discovery even when the resulting actions can be replayed in open loop.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 15 of its 82 references that are in the map. First posted on arXiv 9 Dec 2025.
Builds on
15 of 82 references.
References · 15
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2019Learning by Cheatingcited by 361
- 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 202
- 2016Fast Optical Flow using Dense Inverse Searchcited by 70
- 2024Privileged Sensing Scaffolds Reinforcement Learningcited by 18
- 2018Deep Reinforcement Learning for Event-Triggered Controlcited by 16
- 2025Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flightcited by 12
- 2023Sample-Efficient Learning to Solve a Real-World Labyrinth Game Using Data-Augmented Model-Based Reinforcement Learningcited by 7
- 2024Active Learning for Control-Oriented Identification of Nonlinear Systemscited by 7
- 2021How Are Learned Perception-Based Controllers Impacted by the Limits of Robust Control?cited by 4
- 2016Understanding Robust Control Theory Via Stick Balancingcited by 3
Built on by
Citing papers per year: 2025: 3.
3 papers.