mlsanityData to 7 Oct 2026

Omni-I2C: A Holistic Benchmark for High-Fidelity Image-to-Code Generation

arXiv 2603.17508 ↗First posted 18 Mar 2026cs.CVcited by 3

On the map: Multimodal LLMs / vision-language models → Chart understanding and anomaly detection

Abstract (arXiv)

We present Omni-I2C, a comprehensive benchmark designed to evaluate the capability of Large Multimodal Models (LMMs) in converting complex, structured digital graphics into executable code. We argue that this task represents a non-trivial challenge for the current generation of LMMs: it demands an unprecedented synergy between high-fidelity visual perception -- to parse intricate spatial hierarchies and symbolic details -- and precise generative expression -- to synthesize syntactically sound and logically consistent code. Unlike traditional descriptive tasks, Omni-I2C requires a holistic understanding where any minor perceptual hallucination or coding error leads to a complete failure in visual reconstruction. Omni-I2C features 1080 meticulously curated samples, defined by its breadth across subjects, image modalities, and programming languages. By incorporating authentic user-sourced cases, the benchmark spans a vast spectrum of digital content -- from scientific visualizations to complex symbolic notations -- each paired with executable reference code. To complement this diversity, our evaluation framework provides necessary depth; by decoupling performance into perceptual fidelity and symbolic precision, it transcends surface-level accuracy to expose the granular structural failures and reasoning bottlenecks of current LMMs. Our evaluation reveals a substantial performance gap among leading LMMs; even state-of-the-art models struggle to preserve structural integrity in complex scenarios, underscoring that multimodal code generation remains a formidable challenge. Data and code are available at https://github.com/MiliLab/Omni-I2C.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 33 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

19 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering5 Mar

Other references · 18

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2021Program Synthesis with Large Language Modelscited by 3,509
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2015VQA: Visual Question Answeringcited by 2,489
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  6. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  7. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  8. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  9. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  10. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  12. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 442
6 more
  1. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  2. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  3. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 155
  4. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 54
  5. 2024MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problemscited by 38
  6. 2025Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoningcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generationcited by 3
  2. 2026ShapeCodeBench: A Renewable Benchmark for Perception-to-Program Reconstruction of Synthetic Shape Scenescited by 0
  3. 2026EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025OmniGenBench: A Benchmark for Omnipotent Multimodal Generation across 50+ Tasks
  2. 2026From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs
  3. 2025VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models
  4. 2026Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback
  5. 2024Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots
  6. 2026FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching
  7. 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence
  8. 2026CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation