mlsanityData to 14 Oct 2026

VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learning

arXiv 2511.00391 ↗First posted 1 Nov 2025cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

While recent specialized multimodal code generation models excel in tasks like chart-to-code generation, their reliance on single-task training limits generalization and hinders the development of VIsioN Code Intelligence. In this work, we introduce VinciCoder, a unified framework designed for generalized multimodal code generation. We first curate a large-scale SFT corpus comprising 1.3M direct generation pairs and 300k visual-based refinement tasks. This composition fosters self-refinement capabilities, enabling the model to directly rectify code to align with input images. Subsequently, we propose coarse-to-fine Visual Reinforcement Learning (ViRL) to overcome the brittleness of textual metrics in handling semantically equivalent but syntactically diverse code. By quantifying visual similarity across multi-scale patches, ViRL provides an implementation-agnostic reward mechanism that ensures high-fidelity alignment between rendered outputs and input visuals. Extensive experimental results across diverse benchmarks demonstrate that VinciCoder achieves superior performance, while comprehensive ablation studies validate the effectiveness of our proposed ViRL strategy. The data, code and model are available at https://github.com/DocTron-hub/VinciCoder.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 48 of its 59 references that are in the map. First posted on arXiv 1 Nov 2025.

Builds on

48 of 59 references.

References · 48

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  9. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  10. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  11. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  12. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
36 more
  1. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
  2. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  3. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
  4. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  5. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
  6. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  7. 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 129
  8. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
  9. 2023StarVector: Generating Scalable Vector Graphics Code from Images and Textcited by 102
  10. 2025ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generationcited by 92
  11. 2024Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMscited by 74
  12. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 74
  13. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 74
  14. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
  15. 2025MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoningcited by 45
  16. 2024WebCode2M: A Real-World Dataset for Code Generation from Webpage Designscited by 44
  17. 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluationcited by 42
  18. 2025Rendering-Aware Reinforcement Learning for Vector Graphics Generationcited by 39
  19. 2025ChartEdit: How Far Are MLLMs From Automating Chart Analysis? Evaluating MLLMs' Capability via Chart Editingcited by 32
  20. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedbackcited by 32
  21. 2024Sketch2Code: Evaluating Vision-Language Models for Interactive Web Design Prototypingcited by 29
  22. 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agentscited by 26
  23. 2025Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasonercited by 25
  24. 2024Automatically Generating UI Code from Screenshot: A Divide-and-Conquer-Based Approachcited by 24
  25. 2025TikZero: Zero-Shot Text-Guided Graphics Program Synthesiscited by 24
  26. 2025Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generationcited by 19
  27. 2025MLLM-Based UI2Code Automation Guided by UI Layout Informationcited by 17
  28. 2025SVGenius: Benchmarking LLMs in SVG Understanding, Editing and Generationcited by 16
  29. 2025VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Modelscited by 16
  30. 2025ChartMaster: Advancing Chart-to-Code Generation with Real-World Charts and Chart Similarity Reinforcement Learningcited by 16
  31. 2025UniSVG: A Unified Dataset for Vector Graphic Understanding and Generation with Multimodal Large Language Modelscited by 15
  32. 2025Boosting Chart-to-Code Generation in MLLM via Dual Preference-Guided Refinementcited by 11
  33. 2025ChartM3: Benchmarking Chart Editing with Multimodal Instructionscited by 9
  34. 2025LaTCoder: Converting Webpage Design to Code with Layout-as-Thoughtcited by 9
  35. 2025JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligencecited by 9
  36. 2025OpusAnimation: Code-Based Dynamic Chart Generationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 19.

21 papers.

Sort:

By area:

  1. 2026Multimodal OCR: Parse Anything from Documentscited by 14
  2. 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Modelscited by 12
  3. 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 9
  4. 2025TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Taskscited by 6
  5. 2026Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learningcited by 6
  6. 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
  7. 2026MM-ReCoder: Advancing Chart-to-Code Generation with Reinforcement Learning and Self-Correctioncited by 5
  8. 2026OmniDiagram: Advancing Unified Diagram Code Generation via Visual Interrogation Rewardcited by 4
  9. 2026EchoRL: Reinforcement Learning via Rollout Echoingcited by 4
  10. 2026Visual-ERM: Reward Modeling for Visual Equivalencecited by 3
  11. 2026Training and Agentic Inference Strategies for LLM-based Manim Animation Generationcited by 2
  12. 2026CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolutioncited by 2
9 more
  1. 2026Rubrics as Visual-Repair Context for Self-Evolving UI-to-Code Generationcited by 2
  2. 2025Automating Hardware Design and Verification from Architectural Papers via a Neural-Symbolic Graph Frameworkcited by 1
  3. 2026Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifactscited by 1
  4. 2026GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarioscited by 1
  5. 2026OpenVisTool: An Open Recipe for Synthesizing Instructive Visual Tool-Use Trajectoriescited by 1
  6. 2026ViCo: Visual-oriented Coding with Self-Reflection for Chart Replicationcited by 0
  7. 2026Reinforcement Learning from Intermediate Renders for Image-to-Code Generationcited by 0
  8. 2026PhySimCode: A Benchmark and Evaluation Method for Physics Video to Code Generationcited by 0
  9. 2026PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
  2. 2026CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
  3. 2026Omni-I2C: A Holistic Benchmark for High-Fidelity Image-to-Code Generation
  4. 2025Thinking with Programming Vision: Towards a Unified View for Thinking with Images
  5. 2026Aligning Text, Code, and Vision: A Multi-Objective Reinforcement Learning Framework for Text-to-Visualization
  6. 2026FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching
  7. 2025Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language Models
  8. 2026Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization