# FoldingAgent: Inferring Parametric Origami Procedures from Demonstration Videos

arXiv: https://arxiv.org/abs/2609.00377 · HTML: https://mlsanity.com/paper/2609.00377/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 19 of 19)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation](https://mlsanity.com/paper/2401.04092/) (2024)
- [Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning](https://mlsanity.com/paper/2601.11109/) (2026)
- [A Vision Check-up for Language Models](https://mlsanity.com/paper/2401.01862/) (2024)
- [Re-Thinking Inverse Graphics With Large Language Models](https://mlsanity.com/paper/2404.15228/) (2024)
- [IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Rendering](https://mlsanity.com/paper/2506.23329/) (2025)
- [Gen3DEval: Using vLLMs for Automatic Evaluation of Generated 3D Objects](https://mlsanity.com/paper/2504.08125/) (2025)
- [ORIGAMISPACE: Benchmarking Multimodal LLMs in Multi-Step Spatial Reasoning with Mathematical Constraints](https://mlsanity.com/paper/2511.18450/) (2025)
- [Vision-Language Models as Differentiable Semantic and Spatial Rewards for Text-to-3D Generation](https://mlsanity.com/paper/2509.15772/) (2025)
- [Learn2Fold: Structured Origami Generation with World Model Planning](https://mlsanity.com/paper/2603.29585/) (2026)
- [GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Tasks](https://mlsanity.com/paper/2512.22207/) (2025)
- [Can AI Understand the Language of Origami?](https://mlsanity.com/paper/2603.13856/) (2026)
- [Origami Single-end Capacitive Sensing for Continuous Shape Estimation of Morphing Structures](https://mlsanity.com/paper/2307.05370/) (2023)
- [Rigid Foldability is NP-Hard](https://mlsanity.com/paper/1812.01160/) (2018)
- [An additive algorithm for origami design](https://mlsanity.com/paper/2005.05846/) (2020)
- [Folding Simulation of Rigid Origami with Lagrange Multiplier Method](https://mlsanity.com/paper/2006.05025/) (2020)
- [Harnessing Interpretable Machine Learning for Holistic Inverse Design of Origami](https://mlsanity.com/paper/2204.07235/) (2022)
- [Computational Complexities of Folding](https://mlsanity.com/paper/2410.07666/) (2024)

## Abstract (arXiv)

We present FoldingAgent, an agentic framework for inferring explicit parametric folding programs directly from origami demonstration videos. Our framework leverages the reasoning power of a pre-trained Vision-Language Model (VLM) equipped with a suite of specialized tools that enable the agent to simulate geometric transitions, verify physical plausibility, retrieve and compare visual content, and evaluate its own predictions. To translate visual content into folding programs, we define a parametric space that consists of the paper's geometry and a set of parametric folding actions. Unlike models that predict static crease patterns, our agent operates sequentially and possesses the ability to re-plan its actions, effectively mitigating the compounding errors inherent in multi-step folding. Our approach takes a step toward closing the gap between human origami knowledge, which is primarily shared through unstructured visual demonstrations, and computational methods, which typically rely on structured, parametric representations such as a crease pattern or an executable parametric plan. We evaluate our approach on PurelandFold, a newly curated benchmark of diverse Pureland origami videos with ground-truth geometry and action labels. Our results demonstrate that by combining VLM reasoning with a set of specialized tools and physical simulation, we can successfully transform unstructured visual demonstrations into executable, physically plausible folding procedures.