mlsanityData to 14 Oct 2026

Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances

arXiv 2603.26612 ↗First posted 27 Mar 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Drones equipped with overhead manipulators offer unique capabilities for inspection, maintenance, and contact-based interaction. However, the motion of the drone and its manipulator is tightly linked, and even small attitude changes caused by wind or control imperfections shift the end-effector away from its intended path. This coupling makes reliable tracking difficult and also limits the direct use of learning-based arm controllers that were originally designed for fixed-base robots. These effects appear consistently in our tests whenever the UAV body experiences drift or rapid attitude corrections. To address this behavior, we develop a reinforcement-learning (RL) framework with a transformer-based double deep Q learning (DDQN), with the core idea of using an adaptive beam-search planner that applies a short-horizon beam search over candidate control sequences using the learned critic as the forward estimator. This allows the controller to anticipate the end-effector's motion through simulated rollouts rather than executing those actions directly on the actual model, realizing a software-in-the-loop (SITL) approach. The lookahead relies on value estimates from a Transformer critic that processes short sequences of states, while a DDQN backbone provides the one-step targets needed to keep the learning process stable. Evaluated on a 3-DoF aerial manipulator under identical training conditions, the proposed meta-adaptive planner shows the strongest overall performance with a 10.2% reward increase, a substantial reduction in mean tracking error (from about 6% to 3%), and a 29.6% improvement in the combined reward-error metric relative to the DDQN baseline. Our method exhibits elevated stability in tracking target tip trajectory (by maintaining 5 cm tracking error) when the drone base exhibits drifts due to external disturbances, as opposed to the fixed-beam and Transformer-only variants.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 47 references that are in the map. First posted on arXiv 27 Mar 2026.

Builds on

18 of 47 references.

References · 18

  1. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  2. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  3. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learningcited by 26
  4. 2024A comprehensive survey of research towards AI-enabled unmanned aerial systems in pre-, active-, and post-wildfire managementcited by 25
  5. 2020Nonlinear MPC for Collision Avoidance and Controlof UAVs With Dynamic Obstaclescited by 14
  6. 2023Transformers in Reinforcement Learning: A Surveycited by 12
  7. 2022Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-based Beam Searchcited by 6
  8. 2023Actuator Trajectory Planning for UAVs with Overhead Manipulator using Reinforcement Learningcited by 4
  9. 2025Monte Carlo Beam Search for Actor-Critic Reinforcement Learning in Continuous Controlcited by 3
  10. 2020Aerial Manipulation using Model Predictive Control for Opening a Hinged Doorcited by 2
  11. 2024Adaptive Data Transport Mechanism for UAV Surveillance Missions in Lossy Environmentscited by 2
  12. 2025All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehiclescited by 2
6 more
  1. 2023Static-Equilibrium Oriented Interaction Force Modeling and Control of Aerial Manipulation with Uni-Directional Thrust Multirotorscited by 1
  2. 2023Safe Aerial Manipulator Maneuvering and Force Exertion via Control Barrier Functionscited by 1
  3. 2024ARDDQN: Attention Recurrent Double Deep Q-Network for UAV Coverage Path Planning and Data Harvestingcited by 1
  4. 2024Hybrid Fusion for Battery Degradation Diagnostics Using Minimal Real-World Data: Bridging Laboratory and Practical Applicationscited by 1
  5. 2025Graph Based Deep Reinforcement Learning Aided by Transformers for Multi-Agent Cooperationcited by 1
  6. 2025From Shadow to Light: Toward Safe and Efficient Policy Learning Across MPC, DeePC, RL, and LLM Agentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Global End-Effector Pose Control of an Underactuated Aerial Manipulator via Reinforcement Learning
  2. 2026Physics-Aware End-to-End Deep Reinforcement Learning for Quadcopter Control with Actuator Dynamics
  3. 2025Motion Control in Multi-Rotor Aerial Robots Using Deep Reinforcement Learning
  4. 2026Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty
  5. 2025Leveling the Playing Field: Carefully Comparing Classical and Learned Controllers for Quadrotor Trajectory Tracking
  6. 2025Towards Task-Oriented Flying: Framework, Infrastructure, and Principles
  7. 2024Non-Prehensile Aerial Manipulation using Model-Based Deep Reinforcement Learning
  8. 2025Transformer-Guided Deep Reinforcement Learning for Optimal Takeoff Trajectory Design of an eVTOL Drone