# Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments

arXiv: https://arxiv.org/abs/2506.21129 · HTML: https://mlsanity.com/paper/2506.21129/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 32 of its 52 references that are in the map.
- First posted on arXiv 26 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 30 of 32)

- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Towards Continual Reinforcement Learning: A Review and Perspectives](https://mlsanity.com/paper/2012.13490/) (2020)
- [Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations](https://mlsanity.com/paper/2003.08938/) (2020)
- [Generate To Adapt: Aligning Domains using Generative Adversarial Networks](https://mlsanity.com/paper/1704.01705/) (2017)
- [Adaptive Gradient Methods with Dynamic Bound of Learning Rate](https://mlsanity.com/paper/1902.09843/) (2019)
- [Action Robust Reinforcement Learning and Applications in Continuous Control](https://mlsanity.com/paper/1901.09184/) (2019)
- [Scalable methods for computing state similarity in deterministic Markov Decision Processes](https://mlsanity.com/paper/1911.09291/) (2019)
- [A Sufficient Condition for Convergences of Adam and RMSProp](https://mlsanity.com/paper/1811.09358/) (2018)
- [Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexity](https://mlsanity.com/paper/2208.05767/) (2022)
- [On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgetting](https://mlsanity.com/paper/2206.00761/) (2022)
- [Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning](https://mlsanity.com/paper/2001.09684/) (2020)
- [Finding Mixed Nash Equilibria of Generative Adversarial Networks](https://mlsanity.com/paper/1811.02002/) (2018)
- [Robust Reinforcement Learning via Adversarial training with Langevin Dynamics](https://mlsanity.com/paper/2002.06063/) (2020)
- [Distributionally Robust Learning](https://mlsanity.com/paper/2108.08993/) (2021)
- [Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation](https://mlsanity.com/paper/2102.05714/) (2021)
- [Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptation](https://mlsanity.com/paper/2210.10195/) (2022)
- [Forget Me Not: Reducing Catastrophic Forgetting for Domain Adaptation in Reading Comprehension](https://mlsanity.com/paper/1911.00202/) (2019)
- [Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithms](https://mlsanity.com/paper/2404.03578/) (2024)
- [On the Generalization Gap in Reparameterizable Reinforcement Learning](https://mlsanity.com/paper/1905.12654/) (2019)
- [A Survey on Reinforcement Learning in Aviation Applications](https://mlsanity.com/paper/2211.02147/) (2022)
- [A Dirichlet Process Mixture of Robust Task Models for Scalable Lifelong Reinforcement Learning](https://mlsanity.com/paper/2205.10787/) (2022)
- [Rethinking the Foundations for Continual Reinforcement Learning](https://mlsanity.com/paper/2504.08161/) (2025)
- [Self-Composing Policies for Scalable Continual Reinforcement Learning](https://mlsanity.com/paper/2506.14811/) (2025)
- [Sensor Fusion-based GNSS Spoofing Attack Detection Framework for Autonomous Vehicles](https://mlsanity.com/paper/2106.02982/) (2021)
- [On the Benefit of Optimal Transport for Curriculum Reinforcement Learning](https://mlsanity.com/paper/2309.14091/) (2023)
- [On Corruption-Robustness in Performative Reinforcement Learning](https://mlsanity.com/paper/2505.05609/) (2025)
- [Adaptively Preconditioned Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/1906.04324/) (2019)

## Built on by (most cited first, 1 of 1)

- [Real-Time Bayesian Detection of Drift-Evasive GNSS Spoofing in Reinforcement Learning Based UAV Deconfliction](https://mlsanity.com/paper/2507.11173/) (2025)

## Abstract (arXiv)

Autonomous unmanned aerial vehicles (UAVs) increasingly rely on reinforcement learning (RL) for navigation, yet global navigation satellite system (GNSS) spoofing attacks can induce out-of-distribution observation shifts that corrupt value estimation and degrade mission performance. Existing robust RL methods often lack generalization to unseen attacks. To address this, we propose a curriculum-guided adaptation framework that progressively exposes a robust policy to gradient-based adversarial observation perturbations of increasing intensity while aligning temporal-difference (TD) error distributions across curriculum stages. By preserving TD-error consistency rather than targeting specific attack models, the framework promotes resilience across attack conditions. We further derive a TD-space generalization certificate that bounds performance degradation when the TD-error distribution under a test-time attack remains sufficiently close to that of the final curriculum stage. We evaluate the framework in a UAV deconfliction environment with dynamic 3D obstacles under unseen fixed and dynamic GNSS spoofing attacks. Under fixed spoofing, the curriculum-adapted (CurAd) policy achieves approximately 99-100% mission success across evaluated traffic densities, compared with 20-24% for the best-performing robust baseline, while also outperforming the non-robust vanilla baseline. Under dynamic obstacle-luring spoofing, CurAd achieves the highest episodic rewards and reduces median mission completion steps by up to 22% relative to the best-performing robust RL baseline. These findings demonstrate that adversarial curriculum adaptation transfers resilience from synthetic perturbations to unseen GNSS spoofing attacks without attack-specific retraining, enhancing robust autonomous UAV navigation.