mlsanityData to 14 Oct 2026

Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight

arXiv 2309.09865 ↗First posted 18 Sep 2023cs.RO cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Scene transfer for vision-based mobile robotics applications is a highly relevant and challenging problem. The utility of a robot greatly depends on its ability to perform a task in the real world, outside of a well-controlled lab environment. Existing scene transfer end-to-end policy learning approaches often suffer from poor sample efficiency or limited generalization capabilities, making them unsuitable for mobile robotics applications. This work proposes an adaptive multi-pair contrastive learning strategy for visual representation learning that enables zero-shot scene transfer and real-world deployment. Control policies relying on the embedding are able to operate in unseen environments without the need for finetuning in the deployment environment. We demonstrate the performance of our approach on the task of agile, vision-based quadrotor flight. Extensive simulation and real-world experiments demonstrate that our approach successfully generalizes beyond the training domain and outperforms all baselines.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 52 of its 67 references that are in the map. First posted on arXiv 18 Sep 2023.

Builds on

52 of 67 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  9. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  10. 2022A Generalist Agentcited by 808
  11. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  12. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
38 more
  1. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  2. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  3. 2018Behavioral Cloning from Observationcited by 492
  4. 2020Understanding the Behaviour of Contrastive Losscited by 398
  5. 2019Learning by Cheatingcited by 383
  6. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  7. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  8. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 222
  9. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  10. 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
  11. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 184
  12. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  13. 2016Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planningcited by 150
  14. 2021Learning High-Speed Flight in the Wildcited by 128
  15. 2017Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matchingcited by 125
  16. 2017Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstrationcited by 100
  17. 2019Analyzing and Improving Representations with the Soft Nearest Neighbor Losscited by 95
  18. 2020Flightmare: A Flexible Quadrotor Simulatorcited by 84
  19. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 79
  20. 2020Deep Drone Acrobaticscited by 75
  21. 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 65
  22. 2019Does computer vision matter for action?cited by 64
  23. 2021Autonomous Drone Racing with Deep Reinforcement Learningcited by 60
  24. 2020Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Surveycited by 58
  25. 2018The Double Sphere Camera Modelcited by 52
  26. 2021Learning to See before Learning to Act: Visual Pre-training for Manipulationcited by 51
  27. 2022A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flightcited by 47
  28. 2021Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learningcited by 45
  29. 2022Reinforcement Learning with Neural Radiance Fieldscited by 43
  30. 2021NeuroBEM: Hybrid Aerodynamic Quadrotor Modelcited by 37
  31. 2021Return-Based Contrastive Representation Learning for Reinforcement Learningcited by 36
  32. 2023Autonomous Drone Racing: A Surveycited by 35
  33. 2023Agilicious: Open-Source and Open-Hardware Agile Quadrotor for Vision-Based Flightcited by 30
  34. 2022Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learningcited by 26
  35. 2019Learning to Navigate Using Mid-Level Visual Priorscited by 25
  36. 2018RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Cluttercited by 21
  37. 2020Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentationcited by 21
  38. 2023Autonomous Power Line Inspection with Drones via Perception-Aware MPCcited by 17

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 6.

15 papers.

Sort:

By area:

  1. 2024Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flightcited by 31
  2. 2024NavRL: Learning Safe Flight in Dynamic Environmentscited by 26
  3. 2025The Reality Gap in Robotics: Challenges, Solutions, and Best Practicescited by 24
  4. 2024Multi-Task Reinforcement Learning for Quadrotorscited by 20
  5. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Modelscited by 10
  6. 2024Environment as Policy: Learning to Race in Unseen Trackscited by 6
  7. 2024Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applicationscited by 5
  8. 2025Oracle-Guided Masked Contrastive Reinforcement Learning for Visuomotor Policiescited by 1
  9. 2026Approximate Imitation Learning for Event-based Quadrotor Flight in Cluttered Environmentscited by 1
  10. 2026NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigationcited by 1
  11. 2025HDVIO2.0: Wind and Disturbance Estimation with Hybrid Dynamics VIOcited by 0
  12. 2026Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splattingcited by 0
3 more
  1. 2026GaussFly: Contrastive Reinforcement Learning for Visuomotor Policies in 3D Gaussian Fieldscited by 0
  2. 2026Teacher-Student Representational Alignment for Reinforcement Learning-Driven Imitation Learningcited by 0
  3. 2026Robust Scene Transfer for PointGoal Navigation via Privileged Sensor Guided Contrastive Learningcited by 0