# Reward hacking in physical reinforcement learning revealed by turbulent drag reduction

arXiv: https://arxiv.org/abs/2606.06227 · HTML: https://mlsanity.com/paper/2606.06227/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 63 references that are in the map.
- First posted on arXiv 4 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [Defining and Characterizing Reward Hacking](https://mlsanity.com/paper/2209.13085/) (2022)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies](https://mlsanity.com/paper/2111.02552/) (2021)
- [Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need](https://mlsanity.com/paper/2304.02370/) (2023)
- [Controlling Rayleigh-B'enard convection via Reinforcement Learning](https://mlsanity.com/paper/2003.14358/) (2020)
- [Automating Turbulence Modeling by Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2005.09023/) (2020)
- [GPU acceleration of CaNS for massively-parallel direct numerical simulations of canonical fluid flows](https://mlsanity.com/paper/2001.05234/) (2020)

## Built on by (most cited first, 1 of 1)

- [Gradient-free learning of a closed-loop wall controller for turbulent drag reduction](https://mlsanity.com/paper/2607.12626/) (2026)

## Abstract (arXiv)

Reinforcement-learning controllers optimise specified rewards, but in physical systems those rewards often capture only part of the true control objective. Three mechanisms through which this mismatch can produce apparent success without physical improvement are identified: incomplete accounting that omits relevant costs, constraint enforcement outside the policy that corrupts credit assignment, and observations that fail to resolve the relevant dynamics. All three are demonstrated in active drag reduction of wall-bounded turbulence, where the conservation constraint and full energy budget can be measured directly. A memoryless learnt policy reports drag reduction while raising total dissipation, collapsing to non-physical flow configurations. A recurrent multi-agent controller with the zero-mean projection embedded in the actor, temporal memory matched to the relevant timescales, and an actuation cost that bounds the wall power delivers a physically consistent control. Progress in physical reinforcement learning requires the reward, constraints, observations and evaluation metrics to represent unequivocally the physical objective.