mlsanityData to 7 Oct 2026

Demo2Code: From Summarizing Demonstrations to Synthesizing Code via Extended Chain-of-Thought

arXiv 2305.16744 ↗First posted 26 May 2023cs.ROcited by 39

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Language instructions and demonstrations are two natural ways for users to teach robots personalized tasks. Recent progress in Large Language Models (LLMs) has shown impressive performance in translating language instructions into code for robotic tasks. However, translating demonstrations into task code continues to be a challenge due to the length and complexity of both demonstrations and code, making learning a direct mapping intractable. This paper presents Demo2Code, a novel framework that generates robot task code from demonstrations via an extended chain-of-thought and defines a common latent specification to connect the two. Our framework employs a robust two-stage process: (1) a recursive summarization technique that condenses demonstrations into concise specifications, and (2) a code synthesis approach that expands each function recursively from the generated specifications. We conduct extensive evaluation on various robot task benchmarks, including a novel game benchmark Robotouille, designed to simulate diverse cooking tasks in a kitchen environment. The project's website is available at https://portal-cornell.github.io/demo2code/

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 69 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

7 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Models12 Jul

Other references · 6

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  4. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  5. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  6. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 17; 2025: 6; 2026: 10.

39 papers.

Sort:

By area:

  1. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  2. 2023RoCo: Dialectic Multi-Robot Collaboration with Large Language Modelscited by 171
  3. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
  4. 2024Vision Language Models are In-Context Value Learnerscited by 84
  5. 2024DrEureka: Language Model Guided Sim-To-Real Transfercited by 78
  6. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
  7. 2023SteP: Stacked LLM Policies for Web Actionscited by 66
  8. 2024A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learningcited by 29
  9. 2024Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptionscited by 22
  10. 2024Empowering Large Language Model Agents through Action Learningcited by 20
  11. 2024SwarmBrain: Embodied agent for real-time strategy game StarCraft II via large language modelscited by 18
  12. 2024Eurekaverse: Environment Curriculum Generation via Large Language Modelscited by 14
27 more
  1. 2025ReCode: Unify Plan and Action for Universal Granularity Controlcited by 14
  2. 2024From Pixels to Predicates: Learning Symbolic World Models via Pretrained Vision-Language Modelscited by 11
  3. 2024VLMimic: Vision Language Models are Visual Imitation Learner for Fine-grained Actionscited by 8
  4. 2025Modeling Others' Minds as Codecited by 6
  5. 2023BYOC: Personalized Few-Shot Classification with Co-Authored Class Descriptionscited by 5
  6. 2023O3D: Offline Data-driven Discovery and Distillation for Sequential Decision-Making with Large Language Modelscited by 4
  7. 2024Natural Language as Policies: Reasoning for Coordinate-Level Embodied Control with LLMscited by 4
  8. 2024Knowing What Not to Do: Leverage Language Model Insights for Action Space Pruning in Multi-agent Reinforcement Learningcited by 3
  9. 2025Skill-Aligned Fairness in Multi-Agent Learning for Collaboration in Healthcarecited by 3
  10. 2024DocuMint: Docstring Generation for Python using Small Language Modelscited by 2
  11. 2024RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Modelscited by 2
  12. 2023Program Machine Policy: Addressing Long-Horizon Tasks by Integrating Program Synthesis and State Machinescited by 1
  13. 2025Long-Horizon Visual Imitation Learning via Plan and Code Reflectioncited by 1
  14. 2025Intuitive Programming, Adaptive Task Planning, and Dynamic Role Allocation in Human-Robot Collaborationcited by 1
  15. 2026TMR-VLA:Vision-Language-Action Model for Magnetic Motion Control of Tri-leg Silicone-based Soft Robotcited by 1
  16. 2026A Few Words Go a Long Way: Language Guided Robot Policy Synthesiscited by 1
  17. 2026Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agentscited by 1
  18. 2024Sketch-Plan-Generalize: Learning and Planning with Neuro-Symbolic Programmatic Representations for Inductive Spatial Conceptscited by 0
  19. 2024Creating and Repairing Robot Programs in Open-World Domainscited by 0
  20. 2025Rational Inverse Reasoning: Few-Shot Imitation by Inferring Intent through Planningcited by 0
  21. 2026Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoningcited by 0
  22. 2026Efficient Skill Grounding via Code Refactoring with Small Language Modelscited by 0
  23. 2026Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skillscited by 0
  24. 2026Learning and Transferring Closed-Loop Robot Softwarecited by 0
  25. 2026Generalizing Manipulation Skills with a Local Coding Agentcited by 0
  26. 2026CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulationcited by 0
  27. 2026Encore: Few-Shot Agentic Discovery of Manipulation Strategiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning Novel Skills from Language-Generated Demonstrations
  2. 2025Learning Compositional Behaviors from Demonstration and Language
  3. 2025Growing with Your Embodied Agent: A Human-in-the-Loop Lifelong Code Generation Framework for Long-Horizon Manipulation Skills
  4. 2025LLM Trainer: Automated Robotic Data Generation via Demonstration Augmentation using LLMs
  5. 2026Learning Compositional Symbolic Task Rules from Demonstrations with Inductive Logic Programming
  6. 2025BT-ACTION: A Test-Driven Approach for Modular Understanding of User Instruction Leveraging Behaviour Trees and LLMs ↗
  7. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition
  8. 2025RoboInspector: Unveiling the Unreliability of Policy Code for LLM-enabled Robotic Manipulation