# Forgetful Large Language Models: Lessons Learned from Using LLMs in Robot Programming

arXiv: https://arxiv.org/abs/2310.06646 · HTML: https://mlsanity.com/paper/2310.06646/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2023.

## Built on by (most cited first, 5 of 5)

- [If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents](https://mlsanity.com/paper/2401.00812/) (2024)
- [AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications](https://mlsanity.com/paper/2503.05346/) (2025)
- [AutoMisty: A Multi-Agent LLM Framework for Automated Code Generation in the Misty Social Robot](https://mlsanity.com/paper/2503.06791/) (2025)
- [Teaching AI to Handle Exceptions: Supervised Fine-Tuning with Human-Aligned Judgment](https://mlsanity.com/paper/2503.02976/) (2025)
- [A Grounded Observer Framework for Establishing Guardrails for Foundation Models in Socially Sensitive Domains](https://mlsanity.com/paper/2412.18639/) (2024)

## Abstract (arXiv)

Large language models offer new ways of empowering people to program robot applications-namely, code generation via prompting. However, the code generated by LLMs is susceptible to errors. This work reports a preliminary exploration that empirically characterizes common errors produced by LLMs in robot programming. We categorize these errors into two phases: interpretation and execution. In this work, we focus on errors in execution and observe that they are caused by LLMs being "forgetful" of key information provided in user prompts. Based on this observation, we propose prompt engineering tactics designed to reduce errors in execution. We then demonstrate the effectiveness of these tactics with three language models: ChatGPT, Bard, and LLaMA-2. Finally, we discuss lessons learned from using LLMs in robot programming and call for the benchmarking of LLM-powered end-user development of robot applications.