# Multi-fidelity reinforcement learning framework for shape optimization

arXiv: https://arxiv.org/abs/2202.11170 · HTML: https://mlsanity.com/paper/2202.11170/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 50 references that are in the map.
- First posted on arXiv 22 Feb 2022.

## Builds on (most cited first, 12 of 12)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Direct shape optimization through deep reinforcement learning](https://mlsanity.com/paper/1908.09885/) (2019)
- [Controlling Rayleigh-B'enard convection via Reinforcement Learning](https://mlsanity.com/paper/2003.14358/) (2020)
- [Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows](https://mlsanity.com/paper/2006.02979/) (2020)
- [Learning the aerodynamic design of supercritical airfoils through deep reinforcement learning](https://mlsanity.com/paper/2010.03651/) (2020)
- [Restoring Chaos Using Deep Reinforcement Learning](https://mlsanity.com/paper/1912.00947/) (2019)
- [Computational model discovery with reinforcement learning](https://mlsanity.com/paper/2001.00008/) (2019)
- [Flow Shape Design for Microfluidic Devices Using Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12444/) (2018)

## Built on by (most cited first, 9 of 9)

- [Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer](https://mlsanity.com/paper/2412.09417/) (2024)
- [Transfer learning-enhanced deep reinforcement learning for aerodynamic airfoil optimisation subject to structural constraints](https://mlsanity.com/paper/2505.02634/) (2025)
- [Actively Learning Costly Reward Functions for Reinforcement Learning](https://mlsanity.com/paper/2211.13260/) (2022)
- [A Multi-Fidelity Control Variate Approach for Policy Gradient Estimation](https://mlsanity.com/paper/2503.05696/) (2025)
- [Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systems](https://mlsanity.com/paper/2504.05588/) (2025)
- [Multi-Robot Collaboration through Reinforcement Learning and Abstract Simulation](https://mlsanity.com/paper/2503.05092/) (2025)
- [Transfer learning strategies for accelerating reinforcement-learning-based flow control](https://mlsanity.com/paper/2510.16016/) (2025)
- [Optimization-Embedded Active Multi-Fidelity Surrogate Learning for Multi-Condition Airfoil Shape Optimization](https://mlsanity.com/paper/2603.17057/) (2026)
- [Multi-Fidelity Policy Gradients Stabilize Data-Scarce Reinforcement Learning](https://mlsanity.com/paper/2610.02505/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (DRL) is a promising outer-loop intelligence paradigm which can deploy problem solving strategies for complex tasks. Consequently, DRL has been utilized for several scientific applications, specifically in cases where classical optimization or control methods are limited. One key limitation of conventional DRL methods is their episode-hungry nature which proves to be a bottleneck for tasks which involve costly evaluations of a numerical forward model. In this article, we address this limitation of DRL by introducing a controlled transfer learning framework that leverages a multi-fidelity simulation setting. Our strategy is deployed for an airfoil shape optimization problem at high Reynolds numbers, where our framework can learn an optimal policy for generating efficient airfoil shapes by gathering knowledge from multi-fidelity environments and reduces computational costs by over 30%. Furthermore, our formulation promotes policy exploration and generalization to new environments, thereby preventing over-fitting to data from solely one fidelity. Our results demonstrate this framework's applicability to other scientific DRL scenarios where multi-fidelity environments can be used for policy learning.