UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection
Abstract (arXiv)
Visual-to-Code generation, which transforms scientific plots, vector graphics, and webpages into executable scripts, demands a level of pixel-precise alignment that standard Multimodal Large Language Models (MLLMs) fail to achieve through Supervised Fine-Tuning (SFT) alone. While Reinforcement Learning (RL) offers a theoretical pathway to bridge this gap, its application is hindered by two fundamental obstacles: (1) textitReward Coarseness, where semantic metrics like CLIP scores fail to penalize fine-grained element deviations, and (2) textitExploration Stagnation, where the sparse, heterogeneous code search space prevents the policy from bootstrapping valid trajectories. To overcome these limitations, we introduce UniCoder, a unified RL framework that integrates two novel mechanisms. First, we propose Symbolic Attribute Alignment, which employs a lightweight auxiliary LLM to parse generated code into discrete visual attributes (e.g., hex colors, coordinate limits), enabling dense, element-wise reward computation. Second, to escape local optima, we devise Reference-Guided Code Optimization, a strategy that dynamically injects ground-truth trajectories into low-performing rollout groups, transforming blind exploration into guided policy improvement. Extensive experiments on ChartMimic, UniSVG, Design2Code and ScreenBench benchmarks demonstrate that our 8B-parameter model not only surpasses all open-source baselines but also achieves state-of-the-art performance comparable to proprietary models, establishing a new paradigm for generalized visual-to-code synthesis.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 18 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
14 of 18 references.
Closest ancestors · 4
Same direction, newest first
- 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation19 Aug
- 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs28 Jun
- 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation14 Jun
- 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering5 Mar
Other references · 10
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
- 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
- 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 96
- 2023IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformerscited by 73
- 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 52
- 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agentscited by 23
- 2026Exploring Reasoning Reward Model for Agentscited by 13
- 2026OpenGame: Open Agentic Coding for Gamescited by 12
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinement
- 2025CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
- 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization
- 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generation
- 2026Visual-ERM: Reward Modeling for Visual Equivalence
- 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
- 2026Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts
- 2025UI2Code^N: UI-to-Code Generation as Interactive Visual Optimization