mlsanityData to 7 Oct 2026

UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization

arXiv 2606.31732 ↗First posted 30 Jun 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

Visual-to-Code generation, which transforms scientific plots, vector graphics, and webpages into executable scripts, demands a level of pixel-precise alignment that standard Multimodal Large Language Models (MLLMs) fail to achieve through Supervised Fine-Tuning (SFT) alone. While Reinforcement Learning (RL) offers a theoretical pathway to bridge this gap, its application is hindered by two fundamental obstacles: (1) textitReward Coarseness, where semantic metrics like CLIP scores fail to penalize fine-grained element deviations, and (2) textitExploration Stagnation, where the sparse, heterogeneous code search space prevents the policy from bootstrapping valid trajectories. To overcome these limitations, we introduce UniCoder, a unified RL framework that integrates two novel mechanisms. First, we propose Symbolic Attribute Alignment, which employs a lightweight auxiliary LLM to parse generated code into discrete visual attributes (e.g., hex colors, coordinate limits), enabling dense, element-wise reward computation. Second, to escape local optima, we devise Reference-Guided Code Optimization, a strategy that dynamically injects ground-truth trajectories into low-performing rollout groups, transforming blind exploration into guided policy improvement. Extensive experiments on ChartMimic, UniSVG, Design2Code and ScreenBench benchmarks demonstrate that our 8B-parameter model not only surpasses all open-source baselines but also achieves state-of-the-art performance comparable to proprietary models, establishing a new paradigm for generalized visual-to-code synthesis.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 14 of its 18 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

14 of 18 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation19 Aug
  2. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs28 Jun
  3. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation14 Jun
  4. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering5 Mar

Other references · 10

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  3. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  4. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 157
  5. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 96
  6. 2023IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformerscited by 73
  7. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 52
  8. 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agentscited by 23
  9. 2026Exploring Reasoning Reward Model for Agentscited by 13
  10. 2026OpenGame: Open Agentic Coding for Gamescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinement
  2. 2025CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
  3. 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization
  4. 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generation
  5. 2026Visual-ERM: Reward Modeling for Visual Equivalence
  6. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
  7. 2026Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts
  8. 2025UI2Code^N: UI-to-Code Generation as Interactive Visual Optimization