mlsanityData to 14 Oct 2026

Vision-as-Inverse-Graphics Agent via Interleaved Multimodal Reasoning

arXiv 2601.11109 ↗First posted 16 Jan 2026cs.CV cs.AI cs.GRcited by 49

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Vision-as-inverse-graphics, the concept of reconstructing images into editable programs, remains challenging for Vision-Language Models (VLMs), which inherently lack fine-grained spatial grounding in one-shot settings. To address this, we introduce VIGA (Vision-as-Inverse-Graphics Agent), an interleaved multimodal reasoning framework where symbolic logic and visual perception actively cross-verify each other. VIGA operates through a tightly coupled code-render-inspect loop: synthesizing symbolic programs, projecting them into visual states, and inspecting discrepancies to guide iterative edits. Equipped with high-level semantic skills and an evolving multimodal memory, VIGA sustains evidence-based modifications over long horizons. This training-free, task-agnostic framework seamlessly supports 2D document generation, 3D reconstruction, multi-step 3D editing, and 4D physical interaction. Finally, we introduce BlenderBench, a challenging visual-to-code benchmark. Empirically, VIGA substantially improves accuracy compared with one-shot baselines in BlenderGym (35.32%), SlideBench (117.17%) and our proposed BlenderBench (124.70%).

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 71 references that are in the map. First posted on arXiv 16 Jan 2026.

Builds on

17 of 71 references.

References · 17

  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  2. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  3. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 398
  4. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  5. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
  6. 2025Dynamic Cheatsheet: Test-Time Learning with Adaptive Memorycited by 163
  7. 2025PPTAgent: Generating and Evaluating Presentations Beyond Text-to-Slidescited by 86
  8. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 74
  9. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 67
  10. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 66
  11. 2023Modular Visual Question Answering via Code Generationcited by 48
  12. 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databasescited by 45
5 more
  1. 2025SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Designcited by 24
  2. 2025Task Memory Engine: Spatial Memory for Robust Multi-Step LLM Agentscited by 5
  3. 2025Reflection-Based Memory For Web navigation Agentscited by 5
  4. 2025ALMAS: an Autonomous LLM-based Multi-Agent Software Engineering Frameworkcited by 4
  5. 20253D-Generalist: Self-Improving Vision-Language-Action Models for Crafting 3D Worldscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 49.

49 papers.

Sort:

By area:

  1. 2026GameDevBench: Evaluating Agentic Capabilities Through Game Developmentcited by 28
  2. 20263DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Codecited by 15
  3. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 14
  4. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 14
  5. 2026Thinking in Blender: Staged Executable Inverse Graphics with Vision-Language Modelscited by 10
  6. 2026P3D-Bench: Benchmarking MLLMs for Parametric 3D Generation and Structural Reasoningcited by 9
  7. 2026SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creationcited by 8
  8. 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 8
  9. 2026REST3D: Reconstructing Physically Stable 3D Scenes from a Single Imagecited by 6
  10. 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 6
  11. 2026BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blendercited by 6
  12. 2026Code-as-Room: Generating 3D Rooms from Top-Down View Images via Agentic Code Synthesiscited by 5
37 more
  1. 2026GS-Agent: Creating 4D Physical Worlds With Generative Simulationcited by 5
  2. 2026Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agentscited by 4
  3. 2026PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoningcited by 4
  4. 2026LychSim: A Controllable and Interactive Simulation Framework for Vision Researchcited by 3
  5. 2026Recursive Code World Models: Building Complex Worlds through Recursive Scene Programscited by 3
  6. 20264DCodeBench: Benchmarking Agents on Inverse Graphics of Dynamic Scenescited by 3
  7. 2026WorldAgents: Can Foundation Image Models be Agents for 3D World Models?cited by 2
  8. 2026How Far Are Vision-Language Models from Constructing the Real World? A Benchmark for Physical Generative Reasoningcited by 2
  9. 2026SceneOrchestra: Efficient Agentic 3D Scene Synthesis via Full Tool-Call Trajectory Generationcited by 2
  10. 2026SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestrationcited by 2
  11. 2026PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generationcited by 2
  12. 2026SimEX: Simulation-Integrated Robotics AutoResearchcited by 2
  13. 2026Aggregating LLM-Based Weak Verifiers for Spatial Layout Generationcited by 1
  14. 2026Bunraku: Turning a Single Illustration into an Editable Live2D Charactercited by 1
  15. 2026NeoWorld-Pro: Programming Interactive Scenes from Monocular Images for Embodied Simulationcited by 1
  16. 2026GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learningcited by 1
  17. 20263DHarnessBench: Probing Agentic 3D-to-Code Capabilities of Frontier Vision-Language Modelscited by 1
  18. 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 1
  19. 2026AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videoscited by 1
  20. 2026HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesiscited by 1
  21. 2026DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coachingcited by 0
  22. 2026ArtiMo: Agent-Driven Articulated Mesh Animationcited by 0
  23. 20264DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulationcited by 0
  24. 2026FoldingAgent: Inferring Parametric Origami Procedures from Demonstration Videoscited by 0
  25. 2026Multimodal Thinking with Renderable Programscited by 0
  26. 2026WorldAgent: Verification-Guided Agentic Physical World Constructioncited by 0
  27. 2026CoDimRecon: Agentic Reconstruction of Sim-Ready 3D Scenes with Deformable Curves, Surfaces, and Volumescited by 0
  28. 2026LEGO-Anything: Coding Agents for 3D Scene Reconstructioncited by 0
  29. 2026Asking the World: Generalist Physical Reasoning through Agentic World Modeling and Probingcited by 0
  30. 2026AssemblyWorld: Rethinking 3D Assembly with General-Purpose Agentscited by 0
  31. 2026LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstructioncited by 0
  32. 2026S2PD: Serial-to-Parallel Diffusion for Physically and Logically Consistent Video Generationcited by 0
  33. 2026VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoningcited by 0
  34. 2026S-Space: Exploring Spatial Workspace in Multimodal Language Modelscited by 0
  35. 2026Inverse Garment Design with Visual Agentscited by 0
  36. 2026CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstructioncited by 0
  37. 2026PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstructioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RECODE: Reasoning Through Code Generation for Visual Question Answering
  2. 2026VLM-in-Sandbox: Visual Workspaces for Agentic Visual Reasoning
  3. 2025BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing
  4. 2025VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation
  5. 2025VGR: Visual Grounded Reasoning
  6. 2024BlenderAlchemy: Editing 3D Graphics with Vision-Language Models
  7. 2025Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing
  8. 2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoning