mlsanityData to 7 Oct 2026

Reinforcement Twinning for Hybrid Control of Flapping-Wing Drones

arXiv 2505.18201 ↗First posted 21 May 2025cs.RO cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Controlling flapping-wing drones requires controllers that handle time-varying, nonlinear, underactuated dynamics from incomplete, noisy sensor data. Recent advances in artificial intelligence (AI), particularly reinforcement learning (RL), have opened new perspectives for addressing such complex control problems through data-driven policy optimization from interaction with the environment. Yet purely data-driven methods are sample-inefficient, demanding extensive, sometimes unsafe exploration, especially without guiding physical models. This motivates hybrid AI-physics frameworks. This article proposes a hybrid model-free/model-based flight-control approach using the reinforcement twinning algorithm. The model-based (MB) component uses an adjoint formulation and an adaptive digital twin continuously identified from live trajectories; the model-free (MF) component uses RL. The two agents share knowledge via transfer learning, imitation learning, and shared experience between the real environment and the digital twin, coordinated by a policy referee that selects which agent acts in reality based on digital-twin performance and a real-to-virtual consistency ratio. The framework is evaluated for the longitudinal control of a flapping-wing drone, modelled as a nonlinear time-varying system driven by quasi-steady aerodynamic forces. The hybrid strategy is tested under three adaptive-model initializations: (1) offline identification from existing data, (2) random initialization with fully online identification, and (3) offline pre-training with biased parameters followed by online adaptation. In all cases, the hybrid framework improves performance, robustness, and sample efficiency over purely model-free and purely model-based approaches.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 10 of its 91 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

10 of 91 references.

References · 10

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  4. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  5. 2017Combining Model-Based and Model-Free Updates for Trajectory-Centric Reinforcement Learningcited by 27
  6. 2020Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learningcited by 27
  7. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 22
  8. 2022A Survey on Reinforcement Learning in Aviation Applicationscited by 19
  9. 2020Combining Model-Based and Model-Free Methods for Nonlinear Control: A Provably Convergent Policy Gradient Approachcited by 5
  10. 2019Learning Extreme Hummingbird Maneuvers on Flapping Wing Robotscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Constrained Imitation Learning for a Flapping Wing Unmanned Aerial Vehicle
  2. 2026TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulation
  3. 2019Flying through a narrow gap using neural network: an end-to-end planning and control approach
  4. 2025Dreaming Falcon: Physics-Informed Model-Based Reinforcement Learning for Quadcopters
  5. 2025Hovering Flight of Soft-Actuated Insect-Scale Micro Aerial Vehicles using Deep Reinforcement Learning
  6. 2024Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight
  7. 2025RAPTOR: A Foundation Policy for Quadrotor Control
  8. 2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning