# Enhancing Robot Program Synthesis Through Environmental Context

arXiv: https://arxiv.org/abs/2312.08250 · HTML: https://mlsanity.com/paper/2312.08250/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Dec 2023.

## Built on by (most cited first, 3 of 3)

- [LLMDFA: Analyzing Dataflow in Code with Large Language Models](https://mlsanity.com/paper/2402.10754/) (2024)
- [HumanEval on Latest GPT Models -- 2024](https://mlsanity.com/paper/2402.14852/) (2024)
- [A Neurosymbolic Approach to Adaptive Feature Extraction in SLAM](https://mlsanity.com/paper/2407.06889/) (2024)

## Abstract (arXiv)

Program synthesis aims to automatically generate an executable program that conforms to the given specification. Recent advancements have demonstrated that deep neural methodologies and large-scale pretrained language models are highly proficient in capturing program semantics. For robot programming, prior works have facilitated program synthesis by incorporating global environments. However, the assumption of acquiring a comprehensive understanding of the entire environment is often excessively challenging to achieve. In this work, we present a framework that learns to synthesize a program by rectifying potentially erroneous code segments, with the aid of partially observed environments. To tackle the issue of inadequate attention to partial observations, we propose to first learn an environment embedding space that can implicitly evaluate the impacts of each program token based on the precondition. Furthermore, by employing a graph structure, the model can aggregate both environmental and syntactic information flow and furnish smooth program rectification guidance. Extensive experimental evaluations and ablation studies on the partially observed VizDoom domain authenticate that our method offers superior generalization capability across various tasks and greater robustness when encountering noises.