# Reinforcement Learning for Delivery Drone-Based Participatory Sensing in Dynamic Environments

arXiv: https://arxiv.org/abs/2607.18874 · HTML: https://mlsanity.com/paper/2607.18874/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Online Mixed-Integer Optimization in Milliseconds](https://mlsanity.com/paper/1907.02206/) (2019)
- [Learning Multi-Agent Coordination for Enhancing Target Coverage in Directional Sensor Networks](https://mlsanity.com/paper/2010.13110/) (2020)
- [Hyperspectral Image Super Resolution with Real Unaligned RGB Guidance](https://mlsanity.com/paper/2302.06298/) (2023)
- [AgentSense: LLMs Empower Generalizable and Explainable Web-Based Participatory Urban Sensing](https://mlsanity.com/paper/2510.19661/) (2025)
- [A survey of urban drive-by sensing: An optimization perspective](https://mlsanity.com/paper/2302.00622/) (2023)
- [UAV-Assisted 5G Networks: Mobility-Aware 3D Trajectory Optimization and Resource Allocation for Dynamic Environments](https://mlsanity.com/paper/2504.02613/) (2025)

## Abstract (arXiv)

Using Unmanned Aerial Vehicle (UAV) for urban sensing has emerged as a powerful paradigm to monitor the status of the city, e.g., air quality and noise levels, through agile aerial crowdsourcing. Despite this potential, existing UAV-based sensing approaches overlook environmental disturbances like wind that drastically impact drone velocity and energy efficiency. Consequently, directly applying existing methods to this joint delivery and sensing paradigm in dynamic environments faces two severe challenges: (1) scalability bottlenecks as fleet sizes expand; and (2) multi-timescale decision heterogeneity between macro task dispatching and micro velocity control. To tackle these, we formalize the problem as SensUAV and propose a Two TimeScale Reinforcement Learning framework (TSRL). Specifically, TSRL separates decision-making into two cooperative layers. At the macro level, a task-embedding sensing dispatcher handles scalability by separately encoding distinct task features and sequentially evaluating UAV suitability before task selection. At the micro level, a wind-aware velocity controller learns fine-grained velocity scheduling to adapt to dynamic environmental variations. Extensive experiments on real-world datasets demonstrate that TSRL significantly outperforms baselines, achieving average system profit improvements of 20.1% in Hangzhou and 46.6% in Shanghai.