# BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing

arXiv: https://arxiv.org/abs/2504.01786 · HTML: https://mlsanity.com/paper/2504.01786/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 42 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 30 of 35)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [LayoutGPT: Compositional Visual Planning and Generation with Large Language Models](https://mlsanity.com/paper/2305.15393/) (2023)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [Holodeck: Language Guided Generation of 3D Embodied AI Environments](https://mlsanity.com/paper/2312.09067/) (2023)
- [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023)
- [Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge](https://mlsanity.com/paper/2407.19594/) (2024)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code](https://mlsanity.com/paper/2403.01248/) (2024)
- [Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation](https://mlsanity.com/paper/2401.06591/) (2024)
- [3D-GPT: Procedural 3D Modeling with Large Language Models](https://mlsanity.com/paper/2310.12945/) (2023)
- [AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes](https://mlsanity.com/paper/2312.06644/) (2023)
- [VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation](https://mlsanity.com/paper/2206.08522/) (2022)
- [MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?](https://mlsanity.com/paper/2407.04842/) (2024)
- [The ArtBench Dataset: Benchmarking Generative Models with Artworks](https://mlsanity.com/paper/2206.11404/) (2022)
- [Iterative Motion Editing with Natural Language](https://mlsanity.com/paper/2312.11538/) (2023)
- [LLMR: Real-time Prompting of Interactive Worlds using Large Language Models](https://mlsanity.com/paper/2309.12276/) (2023)
- [Leveraging Large Language Models for NLG Evaluation: Advances and Challenges](https://mlsanity.com/paper/2401.07103/) (2024)
- [Can Large Language Models Understand Symbolic Graphics Programs?](https://mlsanity.com/paper/2408.08313/) (2024)
- [LLplace: The 3D Indoor Scene Layout Generation and Editing via Large Language Model](https://mlsanity.com/paper/2406.03866/) (2024)
- [UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling](https://mlsanity.com/paper/2408.04810/) (2024)
- [L3GO: Language Agents with Chain-of-3D-Thoughts for Generating Unconventional Objects](https://mlsanity.com/paper/2402.09052/) (2024)
- [BlenderAlchemy: Editing 3D Graphics with Vision-Language Models](https://mlsanity.com/paper/2404.17672/) (2024)

## Built on by (most cited first, 30 of 31)

- [HOLODECK 2.0: Vision-Language-Guided 3D World Generation with Editing](https://mlsanity.com/paper/2508.05899/) (2025)
- [3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code](https://mlsanity.com/paper/2606.01057/) (2026)
- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation](https://mlsanity.com/paper/2607.01766/) (2026)
- [BlenderFusion: 3D-Grounded Visual Editing and Generative Compositing](https://mlsanity.com/paper/2506.17450/) (2025)
- [SceneActBench: Can Agents Act on the 3D Scenes They See?](https://mlsanity.com/paper/2607.22393/) (2026)
- [BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender](https://mlsanity.com/paper/2609.15478/) (2026)
- [VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangement](https://mlsanity.com/paper/2512.22351/) (2025)
- [EZBlender: Efficient 3D Editing with Plan-and-ReAct Agent](https://mlsanity.com/paper/2601.07143/) (2026)
- [ServImage: An Image Generation and Editing Benchmark from Real-world Commercial Imaging Services](https://mlsanity.com/paper/2604.24023/) (2026)
- [MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMs](https://mlsanity.com/paper/2607.08970/) (2026)
- [neuralCAD-Edit: An Expert Benchmark for Multimodal-Instructed 3D CAD Model Editing](https://mlsanity.com/paper/2604.16170/) (2026)
- [Deltaynamics: Language-Based Representation for Inferring Rigid-Body Dynamics From Videos](https://mlsanity.com/paper/2605.20576/) (2026)
- [3D Primitives are a Spatial Language for VLMs](https://mlsanity.com/paper/2605.12586/) (2026)
- [PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation](https://mlsanity.com/paper/2607.16355/) (2026)
- [3D-CoS: A New 3D Reconstruction Paradigm Based on VLM Code Synthesis](https://mlsanity.com/paper/2606.10478/) (2026)
- [BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling](https://mlsanity.com/paper/2606.20146/) (2026)
- [3D-DefectBench: A Controlled Factorial Study of Vision-Language Model Evaluation Pipelines for Fine-Grained 3D Generation Defects](https://mlsanity.com/paper/2607.10826/) (2026)
- [3DHarnessBench: Probing Agentic 3D-to-Code Capabilities of Frontier Vision-Language Models](https://mlsanity.com/paper/2609.06535/) (2026)
- [AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos](https://mlsanity.com/paper/2609.24487/) (2026)
- [Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes](https://mlsanity.com/paper/2609.36777/) (2026)
- [Handle-based Mesh Deformation Guided By Vision Language Model](https://mlsanity.com/paper/2506.04562/) (2025)
- [ProcFunc: Function-Oriented Abstractions for Procedural 3D Generation in Python](https://mlsanity.com/paper/2604.26943/) (2026)
- [Running the Gauntlet: Challenging Agentic Tasks](https://mlsanity.com/paper/2606.14397/) (2026)
- [ArtiMo: Agent-Driven Articulated Mesh Animation](https://mlsanity.com/paper/2608.20699/) (2026)
- [EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?](https://mlsanity.com/paper/2609.37686/) (2026)
- [LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstruction](https://mlsanity.com/paper/2610.01863/) (2026)
- [VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning](https://mlsanity.com/paper/2610.13098/) (2026)
- [CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction](https://mlsanity.com/paper/2610.16577/) (2026)
- [PhysCodeGym: Benchmarking Coding Agents for Physically Verifiable 4D Reconstruction](https://mlsanity.com/paper/2610.17288/) (2026)

## Abstract (arXiv)

3D graphics editing is crucial in applications like movie production and game design, yet it remains a time-consuming process that demands highly specialized domain expertise. Automating this process is challenging because graphical editing requires performing a variety of tasks, each requiring distinct skill sets. Recently, vision-language models (VLMs) have emerged as a powerful framework for automating the editing process, but their development and evaluation are bottlenecked by the lack of a comprehensive benchmark that requires human-level perception and presents real-world editing complexity. In this work, we present BlenderGym, the first comprehensive VLM system benchmark for 3D graphics editing. BlenderGym evaluates VLM systems through code-based 3D reconstruction tasks. We evaluate closed- and open-source VLM systems and observe that even the state-of-the-art VLM system struggles with tasks relatively easy for human Blender users. Enabled by BlenderGym, we study how inference scaling techniques impact VLM's performance on graphics editing tasks. Notably, our findings reveal that the verifier used to guide the scaling of generation can itself be improved through inference scaling, complementing recent insights on inference scaling of LLM generation in coding and math tasks. We further show that inference compute is not uniformly effective and can be optimized by strategically distributing it between generation and verification.