# Programming-by-Demonstration for Long-Horizon Robot Tasks

arXiv: https://arxiv.org/abs/2305.03129 · HTML: https://mlsanity.com/paper/2305.03129/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Robot learning from demonstration, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 4 May 2023.

## Placed in

- Offline RL and imitation learning from data / [Robot learning from demonstration](https://mlsanity.com/offlinerl/robot-learning-from-demonstration/)

## Built on by (most cited first, 9 of 9)

- [Agentless: Demystifying LLM-based Software Engineering Agents](https://mlsanity.com/paper/2407.01489/) (2024)
- [CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories](https://mlsanity.com/paper/2502.06111/) (2025)
- [RepoRepair: Leveraging Code Documentation for Repository-Level Automated Program Repair](https://mlsanity.com/paper/2603.01048/) (2026)
- [Meta-RAG on Large Codebases Using Code Summarization](https://mlsanity.com/paper/2508.02611/) (2025)
- [ALMAS: an Autonomous LLM-based Multi-Agent Software Engineering Framework](https://mlsanity.com/paper/2510.03463/) (2025)
- [InterPReT: Interactive Policy Restructuring and Training Enable Effective Imitation Learning from Laypersons](https://mlsanity.com/paper/2602.04213/) (2026)
- [SYNAPSE: SYmbolic Neural-Aided Preference Synthesis Engine](https://mlsanity.com/paper/2403.16689/) (2024)
- [Transformer-XL for Long Sequence Tasks in Robotic Learning from Demonstration](https://mlsanity.com/paper/2405.15562/) (2024)
- [Rational Inverse Reasoning: Few-Shot Imitation by Inferring Intent through Planning](https://mlsanity.com/paper/2508.08983/) (2025)

## Abstract (arXiv)

The goal of programmatic Learning from Demonstration (LfD) is to learn a policy in a programming language that can be used to control a robot's behavior from a set of user demonstrations. This paper presents a new programmatic LfD algorithm that targets long-horizon robot tasks which require synthesizing programs with complex control flow structures, including nested loops with multiple conditionals. Our proposed method first learns a program sketch that captures the target program's control flow and then completes this sketch using an LLM-guided search procedure that incorporates a novel technique for proving unrealizability of programming-by-demonstration problems. We have implemented our approach in a new tool called PROLEX and present the results of a comprehensive experimental evaluation on 120 benchmarks involving complex tasks and environments. We show that, given a 120 second time limit, PROLEX can find a program consistent with the demonstrations in 80% of the cases. Furthermore, for 81% of the tasks for which a solution is returned, PROLEX is able to find the ground truth program with just one demonstration. In comparison, CVC5, a syntax guided synthesis tool, is only able to solve 25% of the cases even when given the ground truth program sketch, and an LLM-based approach, GPT-Synth, is unable to solve any of the tasks due to the environment complexity.