# Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

arXiv: https://arxiv.org/abs/2206.15211 · HTML: https://mlsanity.com/paper/2206.15211/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Model-Based Reinforcement Learning for Atari](https://mlsanity.com/paper/1903.00374/) (2019)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Building machines that adapt and compute like brains](https://mlsanity.com/paper/1711.04203/) (2017)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations](https://mlsanity.com/paper/1909.06993/) (2019)
- [Towards Cognitive Exploration through Deep Reinforcement Learning for Mobile Robots](https://mlsanity.com/paper/1610.01733/) (2016)
- [Deep Reinforcement Learning for Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Medium Transition](https://mlsanity.com/paper/2103.12883/) (2021)
- [Double Critic Deep Reinforcement Learning for Mapless 3D Navigation of Unmanned Aerial Vehicles](https://mlsanity.com/paper/2112.13724/) (2021)
- [Interpretable UAV Collision Avoidance using Deep Reinforcement Learning](https://mlsanity.com/paper/2105.12254/) (2021)

## Built on by (most cited first, 8 of 8)

- [Active Perception Applied To Unmanned Aerial Vehicles Through Deep Reinforcement Learning](https://mlsanity.com/paper/2209.06336/) (2022)
- [BEVNav: Robot Autonomous Navigation Via Spatial-Temporal Contrastive Learning in Bird's-Eye View](https://mlsanity.com/paper/2409.01646/) (2024)
- [Deterministic and Stochastic Analysis of Deep Reinforcement Learning for Low Dimensional Sensing-based Navigation of Mobile Robots](https://mlsanity.com/paper/2209.06328/) (2022)
- [Improving Generalization in Aerial and Terrestrial Mobile Robots Control Through Delayed Policy Learning](https://mlsanity.com/paper/2406.01952/) (2024)
- [Mapless Navigation of a Hybrid Aerial Underwater Vehicle with Deep Reinforcement Learning Through Environmental Generalization](https://mlsanity.com/paper/2209.06332/) (2022)
- [Oracle-Guided Masked Contrastive Reinforcement Learning for Visuomotor Policies](https://mlsanity.com/paper/2510.05692/) (2025)
- [CURLing the Dream: Contrastive Representations for World Modeling in Reinforcement Learning](https://mlsanity.com/paper/2408.05781/) (2024)
- [Monocular Obstacle Avoidance Based on Inverse PPO for Fixed-wing UAVs](https://mlsanity.com/paper/2411.18009/) (2024)

## Abstract (arXiv)

Reinforcement Learning (RL) has presented an impressive performance in video games through raw pixel imaging and continuous control tasks. However, RL performs poorly with high-dimensional observations such as raw pixel images. It is generally accepted that physical state-based RL policies such as laser sensor measurements give a more sample-efficient result than learning by pixels. This work presents a new approach that extracts information from a depth map estimation to teach an RL agent to perform the mapless navigation of Unmanned Aerial Vehicle (UAV). We propose the Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning(Depth-CUPRL) that estimates the depth of images with a prioritized replay memory. We used a combination of RL and Contrastive Learning to lead with the problem of RL based on images. From the analysis of the results with Unmanned Aerial Vehicles (UAVs), it is possible to conclude that our Depth-CUPRL approach is effective for the decision-making and outperforms state-of-the-art pixel-based approaches in the mapless navigation capability.