# Bi-Level Reinforcement Learning Control for an Underactuated Blimp via Center-of-Mass Reconfiguration

arXiv: https://arxiv.org/abs/2605.01289 · HTML: https://mlsanity.com/paper/2605.01289/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 6 of its 38 references that are in the map.
- First posted on arXiv 2 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 6 of 6)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Task and Domain Adaptive Reinforcement Learning for Robot Control](https://mlsanity.com/paper/2404.18713/) (2024)
- [Autonomous Blimp Control via H-infinity Robust Deep Residual Reinforcement Learning](https://mlsanity.com/paper/2303.13929/) (2023)
- [RGBlimp-Q: Robotic Gliding Blimp With Moving Mass Control Based on a Bird-Inspired Continuum Arm](https://mlsanity.com/paper/2406.10810/) (2024)
- [RGBlimp: Robotic Gliding Blimp -- Design, Modeling, Development, and Aerodynamics Analysis](https://mlsanity.com/paper/2306.04079/) (2023)
- [SBlimp: Design, Model, and Translational Motion Control for a Swing-Blimp](https://mlsanity.com/paper/2308.00259/) (2023)

## Abstract (arXiv)

This paper investigates goal-directed tracking control of underactuated blimps with center-of-mass (CoM) reconfiguration. Unlike conventional overactuated blimp designs that rely on redundant actuation for simplified control, this paper focuses on a compact architecture consisting of two thrusters and a movable internal slider, aiming to improve energy efficiency and payload capacity. This hardware-efficient configuration introduces significant underactuation and strong nonlinear coupling between CoM dynamics and vehicle motion. To address these challenges, this paper proposes a bi-level reinforcement learning framework that explicitly decouples task-level CoM planning from continuous thrust control. The outer policy determines a target-dependent CoM configuration prior to flight, while the inner policy generates thrust commands to track straight-line references. To ensure stable learning, this paper introduces a two-stage learning strategy, supported by a convergence analysis of the resulting bi-level process. Extensive simulations and real-world experiments on a 27-goal evaluation set demonstrate that the proposed method consistently outperforms fixed-CoM baselines and PID-based controllers, achieving higher tracking accuracy, enhanced robustness, and reliable sim-to-real transfer.