# Reactive Long Horizon Task Execution via Visual Skill and Precondition Models

arXiv: https://arxiv.org/abs/2011.08694 · HTML: https://mlsanity.com/paper/2011.08694/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2020.

## Built on by (most cited first, 4 of 4)

- [Learning Perceptual Concepts by Bootstrapping from Human Queries](https://mlsanity.com/paper/2111.05251/) (2021)
- [Transporters with Visual Foresight for Solving Unseen Rearrangement Tasks](https://mlsanity.com/paper/2202.10765/) (2022)
- [GePA*SE: Generalized Edge-Based Parallel A* for Slow Evaluations](https://mlsanity.com/paper/2301.10347/) (2023)
- [A-ePA*SE: Anytime Edge-Based Parallel A* for Slow Evaluations](https://mlsanity.com/paper/2305.04408/) (2023)

## Abstract (arXiv)

Zero-shot execution of unseen robotic tasks is important to allowing robots to perform a wide variety of tasks in human environments, but collecting the amounts of data necessary to train end-to-end policies in the real-world is often infeasible. We describe an approach for sim-to-real training that can accomplish unseen robotic tasks using models learned in simulation to ground components of a simple task planner. We learn a library of parameterized skills, along with a set of predicates-based preconditions and termination conditions, entirely in simulation. We explore a block-stacking task because it has a clear structure, where multiple skills must be chained together, but our methods are applicable to a wide range of other problems and domains, and can transfer from simulation to the real-world with no fine tuning. The system is able to recognize failures and accomplish long-horizon tasks from perceptual input, which is critical for real-world execution. We evaluate our proposed approach in both simulation and in the real-world, showing an increase in success rate from 91.6% to 98% in simulation and from 10% to 80% success rate in the real-world as compared with naive baselines. For experiment videos including both real-world and simulation, see: https://www.youtube.com/playlist?list=PL-oD0xHUngeLfQmpngYkGFZarstfPOXqX