# Actuator Trajectory Planning for UAVs with Overhead Manipulator using Reinforcement Learning

arXiv: https://arxiv.org/abs/2308.12843 · HTML: https://mlsanity.com/paper/2308.12843/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 21 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 4 of 4)

- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Aerial Imagery Pile burn detection using Deep Learning: the FLAME dataset](https://mlsanity.com/paper/2012.14036/) (2020)
- [NH-TTC: A gradient-based framework for generalized anticipatory collision avoidance](https://mlsanity.com/paper/1907.05945/) (2019)
- [Mechanical Design, Modelling and Control of a Novel Aerial Manipulator](https://mlsanity.com/paper/1502.07424/) (2015)

## Built on by (most cited first, 4 of 4)

- [A comprehensive survey of research towards AI-enabled unmanned aerial systems in pre-, active-, and post-wildfire management](https://mlsanity.com/paper/2401.02456/) (2024)
- [Monte Carlo Beam Search for Actor-Critic Reinforcement Learning in Continuous Control](https://mlsanity.com/paper/2505.09029/) (2025)
- [Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty](https://mlsanity.com/paper/2606.16621/) (2026)
- [Meta-Adaptive Beam Search Planning for Transformer-Based Reinforcement Learning Control of UAVs with Overhead Manipulators under Flight Disturbances](https://mlsanity.com/paper/2603.26612/) (2026)

## Abstract (arXiv)

In this paper, we investigate the operation of an aerial manipulator system, namely an Unmanned Aerial Vehicle (UAV) equipped with a controllable arm with two degrees of freedom to carry out actuation tasks on the fly. Our solution is based on employing a Q-learning method to control the trajectory of the tip of the arm, also called end-effector. More specifically, we develop a motion planning model based on Time To Collision (TTC), which enables a quadrotor UAV to navigate around obstacles while ensuring the manipulator's reachability. Additionally, we utilize a model-based Q-learning model to independently track and control the desired trajectory of the manipulator's end-effector, given an arbitrary baseline trajectory for the UAV platform. Such a combination enables a variety of actuation tasks such as high-altitude welding, structural monitoring and repair, battery replacement, gutter cleaning, skyscrapper cleaning, and power line maintenance in hard-to-reach and risky environments while retaining compatibility with flight control firmware. Our RL-based control mechanism results in a robust control strategy that can handle uncertainties in the motion of the UAV, offering promising performance. Specifically, our method achieves 92% accuracy in terms of average displacement error (i.e. the mean distance between the target and obtained trajectory points) using Q-learning with 15,000 episodes