# Learning Vision-based Reactive Policies for Obstacle Avoidance

arXiv: https://arxiv.org/abs/2010.16298 · HTML: https://mlsanity.com/paper/2010.16298/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2020.

## Built on by (most cited first, 12 of 12)

- [On the Role of the Action Space in Robot Manipulation Learning and Sim-to-Real Transfer](https://mlsanity.com/paper/2312.03673/) (2023)
- [Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning](https://mlsanity.com/paper/2105.04962/) (2021)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [Accelerating Model-Based Reinforcement Learning with State-Space World Models](https://mlsanity.com/paper/2502.20168/) (2025)
- [RMP2: A Structured Composable Policy Class for Robot Learning](https://mlsanity.com/paper/2103.05922/) (2021)
- [Reactive Motion Generation on Learned Riemannian Manifolds](https://mlsanity.com/paper/2203.07761/) (2022)
- [CLAS: Coordinating Multi-Robot Manipulation with Central Latent Action Spaces](https://mlsanity.com/paper/2211.15824/) (2022)
- [Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning](https://mlsanity.com/paper/2110.00784/) (2021)
- [CLAMGen: Closed-Loop Arm Motion Generation via Multi-view Vision-Based RL](https://mlsanity.com/paper/2103.13267/) (2021)
- [The Shortcomings of Force-from-Motion in Robot Learning](https://mlsanity.com/paper/2407.02904/) (2024)
- [Dual-Arm Adversarial Robot Learning](https://mlsanity.com/paper/2110.08066/) (2021)
- [Fast Trajectory Planner with a Reinforcement Learning-based Controller for Robotic Manipulators](https://mlsanity.com/paper/2509.17381/) (2025)

## Abstract (arXiv)

In this paper, we address the problem of vision-based obstacle avoidance for robotic manipulators. This topic poses challenges for both perception and motion generation. While most work in the field aims at improving one of those aspects, we provide a unified framework for approaching this problem. The main goal of this framework is to connect perception and motion by identifying the relationship between the visual input and the corresponding motion representation. To this end, we propose a method for learning reactive obstacle avoidance policies. We evaluate our method on goal-reaching tasks for single and multiple obstacles scenarios. We show the ability of the proposed method to efficiently learn stable obstacle avoidance strategies at a high success rate, while maintaining closed-loop responsiveness required for critical applications like human-robot interaction.