# Pretty darn good control: when are approximate solutions better than approximate models

arXiv: https://arxiv.org/abs/2308.13654 · HTML: https://mlsanity.com/paper/2308.13654/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 42 references that are in the map.
- First posted on arXiv 25 Aug 2023.

## Builds on (most cited first, 7 of 7)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [A Survey on Model-based Reinforcement Learning](https://mlsanity.com/paper/2206.09328/) (2022)
- [Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey](https://mlsanity.com/paper/1904.04973/) (2019)
- [Deep Reinforcement Learning for Conservation Decisions](https://mlsanity.com/paper/2106.08272/) (2021)
- [Bridging adaptive management and reinforcement learning for more robust decisions](https://mlsanity.com/paper/2303.08731/) (2023)

## Built on by (most cited first, 1 of 1)

- [Using machine learning to inform harvest control rule design in complex fishery settings](https://mlsanity.com/paper/2412.12400/) (2024)

## Abstract (arXiv)

Existing methods for optimal control struggle to deal with the complexity commonly encountered in real-world systems, including dimensionality, process error, model bias and data heterogeneity. Instead of tackling these system complexities directly, researchers have typically sought to simplify models to fit optimal control methods. But when is the optimal solution to an approximate, stylized model better than an approximate solution to a more accurate model? While this question has largely gone unanswered owing to the difficulty of finding even approximate solutions for complex models, recent algorithmic and computational advances in deep reinforcement learning (DRL) might finally allow us to address these questions. DRL methods have to date been applied primarily in the context of games or robotic mechanics, which operate under precisely known rules. Here, we demonstrate the ability for DRL algorithms using deep neural networks to successfully approximate solutions (the "policy function" or control rule) in a non-linear three-variable model for a fishery without knowing or ever attempting to infer a model for the process itself. We find that the reinforcement learning agent discovers an effective simplification of the problem to obtain an interpretable control rule. We show that the policy obtained with DRL is both more profitable and more sustainable than any constant mortality policy -- the standard family of policies considered in fishery management.