# VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learning

arXiv: https://arxiv.org/abs/2511.00391 · HTML: https://mlsanity.com/paper/2511.00391/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 59 references that are in the map.
- First posted on arXiv 1 Nov 2025.

## Builds on (most cited first, 30 of 48)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025)
- [Visual-RFT: Visual Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.01785/) (2025)
- [R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization](https://mlsanity.com/paper/2503.10615/) (2025)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement](https://mlsanity.com/paper/2503.06520/) (2025)
- [Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering](https://mlsanity.com/paper/2403.03163/) (2024)
- [Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset](https://mlsanity.com/paper/2403.09029/) (2024)
- [ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation](https://mlsanity.com/paper/2406.09961/) (2024)
- [StarVector: Generating Scalable Vector Graphics Code from Images and Text](https://mlsanity.com/paper/2312.11556/) (2023)
- [ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation](https://mlsanity.com/paper/2501.06598/) (2025)
- [Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs](https://mlsanity.com/paper/2406.20098/) (2024)
- [OmniSVG: A Unified Scalable Vector Graphics Generation Model](https://mlsanity.com/paper/2504.06263/) (2025)
- [GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents](https://mlsanity.com/paper/2505.15810/) (2025)
- [Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation](https://mlsanity.com/paper/2502.14846/) (2025)
- [MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning](https://mlsanity.com/paper/2505.10557/) (2025)
- [WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs](https://mlsanity.com/paper/2404.06369/) (2024)
- [ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluation](https://mlsanity.com/paper/2507.04952/) (2025)
- [Rendering-Aware Reinforcement Learning for Vector Graphics Generation](https://mlsanity.com/paper/2505.20793/) (2025)

## Built on by (most cited first, 21 of 21)

- [Multimodal OCR: Parse Anything from Documents](https://mlsanity.com/paper/2603.13032/) (2026)
- [OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Models](https://mlsanity.com/paper/2601.21639/) (2026)
- [VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generation](https://mlsanity.com/paper/2602.05998/) (2026)
- [TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks](https://mlsanity.com/paper/2511.06283/) (2025)
- [Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning](https://mlsanity.com/paper/2604.06079/) (2026)
- [Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence](https://mlsanity.com/paper/2606.15932/) (2026)
- [MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction](https://mlsanity.com/paper/2604.01600/) (2026)
- [OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward](https://mlsanity.com/paper/2604.05514/) (2026)
- [EchoRL: Reinforcement Learning via Rollout Echoing](https://mlsanity.com/paper/2605.31228/) (2026)
- [Visual-ERM: Reward Modeling for Visual Equivalence](https://mlsanity.com/paper/2603.13224/) (2026)
- [Training and Agentic Inference Strategies for LLM-based Manim Animation Generation](https://mlsanity.com/paper/2604.18364/) (2026)
- [CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution](https://mlsanity.com/paper/2604.22192/) (2026)
- [Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation](https://mlsanity.com/paper/2608.24138/) (2026)
- [Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Framework](https://mlsanity.com/paper/2511.06067/) (2025)
- [Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts](https://mlsanity.com/paper/2606.10334/) (2026)
- [GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios](https://mlsanity.com/paper/2607.28073/) (2026)
- [OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories](https://mlsanity.com/paper/2608.08557/) (2026)
- [ViCo: Visual-oriented Coding with Self-Reflection for Chart Replication](https://mlsanity.com/paper/2609.16014/) (2026)
- [Reinforcement Learning from Intermediate Renders for Image-to-Code Generation](https://mlsanity.com/paper/2609.34587/) (2026)
- [PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generation](https://mlsanity.com/paper/2610.15722/) (2026)
- [PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis](https://mlsanity.com/paper/2610.20538/) (2026)

## Abstract (arXiv)

While recent specialized multimodal code generation models excel in tasks like chart-to-code generation, their reliance on single-task training limits generalization and hinders the development of VIsioN Code Intelligence. In this work, we introduce VinciCoder, a unified framework designed for generalized multimodal code generation. We first curate a large-scale SFT corpus comprising 1.3M direct generation pairs and 300k visual-based refinement tasks. This composition fosters self-refinement capabilities, enabling the model to directly rectify code to align with input images. Subsequently, we propose coarse-to-fine Visual Reinforcement Learning (ViRL) to overcome the brittleness of textual metrics in handling semantically equivalent but syntactically diverse code. By quantifying visual similarity across multi-scale patches, ViRL provides an implementation-agnostic reward mechanism that ensures high-fidelity alignment between rendered outputs and input visuals. Extensive experimental results across diverse benchmarks demonstrate that VinciCoder achieves superior performance, while comprehensive ablation studies validate the effectiveness of our proposed ViRL strategy. The data, code and model are available at https://github.com/DocTron-hub/VinciCoder.