# Co-Design of Aeroelastic Systems with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2607.15329 · HTML: https://mlsanity.com/paper/2607.15329/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 60 references that are in the map.
- First posted on arXiv 16 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [Benchmarking Reinforcement Learning Algorithms on Real-World Robots](https://mlsanity.com/paper/1809.07731/) (2018)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.04460/) (2020)
- [A Review of Nine Physics Engines for Reinforcement Learning Research](https://mlsanity.com/paper/2407.08590/) (2024)
- [PyFlyt -- UAV Simulation Environments for Reinforcement Learning Research](https://mlsanity.com/paper/2304.01305/) (2023)
- [N-LIMB: Neural Limb Optimization for Efficient Morphological Design](https://mlsanity.com/paper/2207.11773/) (2022)
- [Co-Design of Autonomous Systems: From Hardware Selection to Control Synthesis](https://mlsanity.com/paper/2011.10758/) (2020)
- [Co-Design Optimisation of Morphing Topology and Control of Winged Drones](https://mlsanity.com/paper/2309.13948/) (2023)

## Abstract (arXiv)

Control co-design considers the physical system and its controller together, enabling the strong coupling between system design and control to be uncovered and exploited. This is especially relevant in aeroelastic flight systems, where structural, aerodynamic, and control design choices jointly determine manoeuvrability and efficiency. This paper presents a model-free nested co-design framework for aeroelastic systems using deep reinforcement learning, in which a design-conditioned control policy is trained with proximal policy optimisation while an outer loop updates a distribution over candidate design parameters. The approach is evaluated on three case studies of increasing complexity: a spring-mass-damper system, a pitch-plunge-flap aerofoil, and a highly flexible high-aspect-ratio glider performing a thermal-soaring mission in a stochastic environment. Across these case studies, the framework is shown to progressively concentrate the design search towards high-performing regions and to outperform policies trained on randomly sampled designs. The results also show that reward shaping plays an important role in enabling stable learning in partially observed and stochastic environments. In the final glider case, the method jointly addresses wing design, flight control, and mission-level behaviour in the presence of aeroelastic coupling and atmospheric uncertainty. These results highlight the potential of model-free co-design for complex aeroelastic systems in which design, control, and mission objectives are tightly coupled.