VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
While recent specialized multimodal code generation models excel in tasks like chart-to-code generation, their reliance on single-task training limits generalization and hinders the development of VIsioN Code Intelligence. In this work, we introduce VinciCoder, a unified framework designed for generalized multimodal code generation. We first curate a large-scale SFT corpus comprising 1.3M direct generation pairs and 300k visual-based refinement tasks. This composition fosters self-refinement capabilities, enabling the model to directly rectify code to align with input images. Subsequently, we propose coarse-to-fine Visual Reinforcement Learning (ViRL) to overcome the brittleness of textual metrics in handling semantically equivalent but syntactically diverse code. By quantifying visual similarity across multi-scale patches, ViRL provides an implementation-agnostic reward mechanism that ensures high-fidelity alignment between rendered outputs and input visuals. Extensive experimental results across diverse benchmarks demonstrate that VinciCoder achieves superior performance, while comprehensive ablation studies validate the effectiveness of our proposed ViRL strategy. The data, code and model are available at https://github.com/DocTron-hub/VinciCoder.
Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 48 of its 59 references that are in the map. First posted on arXiv 1 Nov 2025.
Builds on
48 of 59 references.
References · 48
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
36 more
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
- 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
- 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
- 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
- 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
- 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 102
- 2025ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generationcited by 92
- 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
- 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 74
- 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 74
- 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
- 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
- 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 44
- 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluationcited by 42
- 2025Rendering-Aware Reinforcement Learning for Vector Graphics Generationcited by 39
- 2025ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editingcited by 32
- 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 32
- 2024Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototypingcited by 29
- 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agentscited by 26
- 2025Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasonercited by 25
- 2024Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approachcited by 24
- 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 24
- 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generationcited by 19
- 2025MLLM-Based UI2Code Automation Guided by UI Layout Informationcited by 17
- 2025SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generationcited by 16
- 2025VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Modelscited by 16
- 2025ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learningcited by 16
- 2025UniSVG: A Unified Dataset for Vector Graphic Understanding and Generation with Multimodal Large Language Modelscited by 15
- 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinementcited by 11
- 2025ChartM3: Benchmarking Chart Editing with Multimodal Instructionscited by 9
- 2025LaTCoder: Converting Webpage Design to Code with Layout-as-Thoughtcited by 9
- 2025JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligencecited by 9
- 2025OpusAnimation: Code-Based Dynamic Chart Generationcited by 3
Built on by
Citing papers per year: 2025: 2; 2026: 19.
21 papers.
Sort:
By area:
- 2026Multimodal OCR: Parse Anything from Documentscited by 14
- 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Modelscited by 12
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 9
- 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Taskscited by 6
- 2026Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learningcited by 6
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
- 2026MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correctioncited by 5
- 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Rewardcited by 4
- 2026EchoRL: Reinforcement Learning via Rollout Echoingcited by 4
- 2026Visual-ERM: Reward Modeling for Visual Equivalencecited by 3
- 2026Training and Agentic Inference Strategies for LLM-based Manim Animation Generationcited by 2
- 2026CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolutioncited by 2
9 more
- 2026Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generationcited by 2
- 2025Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Frameworkcited by 1
- 2026Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifactscited by 1
- 2026GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarioscited by 1
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectoriescited by 1
- 2026ViCo: Visual-oriented Coding with Self-Reflection for Chart Replicationcited by 0
- 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generationcited by 0
- 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generationcited by 0
- 2026PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesiscited by 0
- 2026PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis14 Oct 2026
- 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generation11 Oct 2026
- 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generation28 Sep 2026
- 2026ViCo: Visual-oriented Coding with Self-Reflection for Chart Replication31 Aug 2026
- 2026Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation25 Aug 2026
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories9 Aug 2026
- 2026GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios30 Jul 2026
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence14 Jun 2026
- 2026Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifacts9 Jun 2026
- 2026EchoRL: Reinforcement Learning via Rollout Echoing29 May 2026
- 2026CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution24 Apr 2026
- 2026Training and Agentic Inference Strategies for LLM-based Manim Animation Generation20 Apr 2026
9 more
- 2026Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning7 Apr 2026
- 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward7 Apr 2026
- 2026MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction2 Apr 2026
- 2026Visual-ERM: Reward Modeling for Visual Equivalence13 Mar 2026
- 2026Multimodal OCR: Parse Anything from Documents13 Mar 2026
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generation5 Feb 2026
- 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Models29 Jan 2026
- 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks9 Nov 2025
- 2025Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Framework8 Nov 2025
- 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Modelscited by 12
- 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Taskscited by 6
- 2026MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correctioncited by 5
- 2026CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolutioncited by 2
- 2026Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generationcited by 2
- 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generationcited by 0
- 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generationcited by 0
- 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generation11 Oct 2026
- 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generation28 Sep 2026
- 2026Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generation25 Aug 2026
- 2026CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution24 Apr 2026
- 2026MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correction2 Apr 2026
- 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Models29 Jan 2026
- 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks9 Nov 2025
- 2026Visual-ERM: Reward Modeling for Visual Equivalencecited by 3
- 2026Visual-ERM: Reward Modeling for Visual Equivalence13 Mar 2026
- 2026Multimodal OCR: Parse Anything from Documentscited by 14
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 9
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
- 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Rewardcited by 4
- 2025Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Frameworkcited by 1
- 2026GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarioscited by 1
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectoriescited by 1
- 2026ViCo: Visual-oriented Coding with Self-Reflection for Chart Replicationcited by 0
- 2026PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesiscited by 0
- 2026PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis14 Oct 2026
- 2026ViCo: Visual-oriented Coding with Self-Reflection for Chart Replication31 Aug 2026
- 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectories9 Aug 2026
- 2026GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios30 Jul 2026
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence14 Jun 2026
- 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Reward7 Apr 2026
- 2026Multimodal OCR: Parse Anything from Documents13 Mar 2026
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generation5 Feb 2026
- 2025Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Framework8 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2026UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
- 2026CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
- 2026Omni-I2C: A Holistic Benchmark for High-Fidelity Image-to-Code Generation
- 2025Thinking with Programming Vision: Towards a Unified View for Thinking with Images
- 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization
- 2026FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching
- 2025Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language Models
- 2026Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization