# Dreaming Falcon: Physics-Informed Model-Based Reinforcement Learning for Quadcopters

arXiv: https://arxiv.org/abs/2511.18243 · HTML: https://mlsanity.com/paper/2511.18243/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Latent dynamics models and model-based RL, one of 11 directions in World models.
- Builds on 4 of its 5 references that are in the map.
- First posted on arXiv 23 Nov 2025.

## Placed in

- World models / [Latent dynamics models and model-based RL](https://mlsanity.com/wm/latent-dynamics-models-and-model-based-rl/)
- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 4 of 4)

- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Multi-Step Prediction of Dynamic Systems with Recurrent Neural Networks](https://mlsanity.com/paper/1806.00526/) (2018)
- [Physics-Informed Model-Based Reinforcement Learning](https://mlsanity.com/paper/2212.02179/) (2022)

## Built on by (most cited first, 1 of 1)

- [ClearNav: Perception-Aligned Quadrotor Flight Using Differential-Physics](https://mlsanity.com/paper/2610.17419/) (2026)

## Abstract (arXiv)

Current control algorithms for aerial robots struggle with robustness in dynamic environments and adverse conditions. Model-based reinforcement learning (RL) has shown strong potential in handling these challenges while remaining sample-efficient. Additionally, Dreamer has demonstrated that online model-based RL can be achieved using a recurrent world model trained on replay buffer data. However, applying Dreamer to aerial systems has been quite challenging due to its sample inefficiency and poor generalization of dynamics models. Our work explores a physics-informed approach to world model learning and improves policy performance. The world model treats the quadcopter as a free-body system and predicts the net forces and moments acting on it, which are then passed through a 6-DOF Runge-Kutta integrator (RK4) to predict future state rollouts. In this paper, we compare this physics-informed method to a standard RNN-based world model. Although both models perform well on the training data, we observed that they fail to generalize to new trajectories, leading to rapid divergence in state rollouts, preventing policy convergence.