# Data Freshness and Energy-Efficient UAV Navigation Optimization: A Deep Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2003.04816 · HTML: https://mlsanity.com/paper/2003.04816/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2020.

## Built on by (most cited first, 9 of 9)

- [Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary Survey](https://mlsanity.com/paper/2211.04324/) (2022)
- [Semantics-Empowered Communication: A Tutorial-cum-Survey](https://mlsanity.com/paper/2212.08487/) (2022)
- [Autonomous Unmanned Aerial Vehicle Navigation using Reinforcement Learning: A Systematic Review](https://mlsanity.com/paper/2208.12328/) (2022)
- [UAV Trajectory Planning for AoI-Minimal Data Collection in UAV-Aided IoT Networks by Transformer](https://mlsanity.com/paper/2401.02425/) (2023)
- [DeepQTest: Testing Autonomous Driving Systems with Reinforcement Learning and Real-world Weather Data](https://mlsanity.com/paper/2310.05170/) (2023)
- [Energy-Aware Multi-Agent Reinforcement Learning for Collaborative Execution in Mission-Oriented Drone Networks](https://mlsanity.com/paper/2410.22578/) (2024)
- [Optimizing the Long-Term Average Reward for Continuing MDPs: A Technical Report](https://mlsanity.com/paper/2104.06139/) (2021)
- [Joint Cluster Head Selection and Trajectory Planning in UAV-Aided IoT Networks by Reinforcement Learning with Sequential Model](https://mlsanity.com/paper/2112.00333/) (2021)
- [A Survey of Freshness-Aware Wireless Networking with Reinforcement Learning](https://mlsanity.com/paper/2512.21412/) (2025)

## Abstract (arXiv)

In this paper, we design a navigation policy for multiple unmanned aerial vehicles (UAVs) where mobile base stations (BSs) are deployed to improve the data freshness and connectivity to the Internet of Things (IoT) devices. First, we formulate an energy-efficient trajectory optimization problem in which the objective is to maximize the energy efficiency by optimizing the UAV-BS trajectory policy. We also incorporate different contextual information such as energy and age of information (AoI) constraints to ensure the data freshness at the ground BS. Second, we propose an agile deep reinforcement learning with experience replay model to solve the formulated problem concerning the contextual constraints for the UAV-BS navigation. Moreover, the proposed approach is well-suited for solving the problem, since the state space of the problem is extremely large and finding the best trajectory policy with useful contextual features is too complex for the UAV-BSs. By applying the proposed trained model, an effective real-time trajectory policy for the UAV-BSs captures the observable network states over time. Finally, the simulation results illustrate the proposed approach is 3.6% and 3.13% more energy efficient than those of the greedy and baseline deep Q Network (DQN) approaches.