# Imperfect Visual Verification for Code Edition : A Case Study on TikZ

arXiv: https://arxiv.org/abs/2606.15693 · HTML: https://mlsanity.com/paper/2606.15693/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 30 of 31)

- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://mlsanity.com/paper/2305.11738/) (2023)
- [Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models](https://mlsanity.com/paper/2405.01535/) (2024)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs](https://mlsanity.com/paper/2406.01297/) (2024)
- [Is Self-Repair a Silver Bullet for Code Generation?](https://mlsanity.com/paper/2306.09896/) (2023)
- [Self-Edit: Fault-Aware Code Editor for Code Generation](https://mlsanity.com/paper/2305.04087/) (2023)
- [StarVector: Generating Scalable Vector Graphics Code from Images and Text](https://mlsanity.com/paper/2312.11556/) (2023)
- [Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision](https://mlsanity.com/paper/2311.07362/) (2023)
- [Shepherd: A Critic for Language Model Generation](https://mlsanity.com/paper/2308.04592/) (2023)
- [ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation](https://mlsanity.com/paper/2507.04952/) (2025)
- [ALGO: Synthesizing Algorithmic Programs with LLM-Generated Oracle Verifiers](https://mlsanity.com/paper/2305.14591/) (2023)
- [METAL: A Multi-Agent Framework for Chart Generation with Test-Time Scaling](https://mlsanity.com/paper/2502.17651/) (2025)
- [Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning](https://mlsanity.com/paper/2511.19900/) (2025)
- [ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents](https://mlsanity.com/paper/2507.22827/) (2025)
- [PaperBanana: Automating Academic Illustration for AI Scientists](https://mlsanity.com/paper/2601.23265/) (2026)
- [Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization](https://mlsanity.com/paper/2406.00507/) (2024)
- [VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models](https://mlsanity.com/paper/2508.09945/) (2025)
- [VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual Representation](https://mlsanity.com/paper/2511.02778/) (2025)
- [Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers](https://mlsanity.com/paper/2401.01974/) (2024)
- [CriticEval: Evaluating Large Language Model as Critic](https://mlsanity.com/paper/2402.13764/) (2024)
- [RealCritic: Towards Effectiveness-Driven Evaluation of Language Model Critiques](https://mlsanity.com/paper/2501.14492/) (2025)
- [VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation](https://mlsanity.com/paper/2506.13326/) (2025)
- [Improved Iterative Refinement for Chart-to-Code Generation via Structured Instruction](https://mlsanity.com/paper/2506.14837/) (2025)
- [Multimodal Auto Validation For Self-Refinement in Web Agents](https://mlsanity.com/paper/2410.00689/) (2024)
- [LLM Code Customization with Visual Results: A Benchmark on TikZ](https://mlsanity.com/paper/2505.04670/) (2025)
- [SR-Eval: Evaluating LLMs on Code Generation under Stepwise Requirement Refinement](https://mlsanity.com/paper/2509.18808/) (2025)
- [VF-Eval: Evaluating Multimodal LLMs for Generating Feedback on AIGC Videos](https://mlsanity.com/paper/2505.23693/) (2025)

## Abstract (arXiv)

LLMs have significantly advanced code generation, enabling the synthesis of functional programs. While recent systems achieve strong performance on many coding benchmarks, tasks involving programs such as TikZ that generate visual artifacts remain challenging, in particular on visual code customization. Unlike generation from scratch, customization requires localized, semantics-preserving edits: the model must locate relevant code, modify it according to the instruction, and preserve the remaining structure and rendering. Approaches based on post-hoc iterative refinement/correction where a verifier provides feedback to guide corrections, have shown promise. However, in the case of programs with a visual outcome such as in TikZ, where correctness is harder or likely impossible to formalize and evaluate automatically, deterministic verifiers do not exist. Hence, developers can only rely on imperfect verifiers. In this paper, we conduct an empirical study to answer:to what extent can iterative refinement remain effective when the verifier itself is unreliable?} We use TikZ as a focused case study that isolates the core difficulties of the problem (weak code structure, fine-grained visual semantics, and difficult feature localization) in a controlled and challenging setting. We define visual code customization as an iterative editing problem with an imperfect oracle, and introduce a framework for analyzing such iterative refinements. We conduct a large-scale study and evaluate multiple LLM-based and tool-augmented visual verifiers within iterative refinement pipelines, and perform extensive manual annotation of refinement trajectories to assess verifier behavior and feedback quality. Our findings show that even imperfect verifiers can determine with moderate accuracy whether visual instructions are applied to code, achieving F1-scores up to 0.815. Feedback improves iterative refinement, especially for weaker models, adding 11--20 perfect customizations for Qwen3-vl-30b-a3b-Instruct, while stronger models like Gemini-3 gain fewer improvements (+5) but benefit more from accurate verification that prevents premature acceptance. Feedback is effective only when it precisely identifies image issues, provides actionable guidance, addresses all relevant problems, and remains grounded in the original instruction.