mlsanityData to 7 Oct 2026

AI2D-RST: A multimodal corpus of 1000 primary school science diagrams

arXiv 1912.03879 ↗First posted 9 Dec 2019cs.CL cs.CVcited by 60

Not on the map. Why?

Cited by 36 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This article introduces AI2D-RST, a multimodal corpus of 1000 English-language diagrams that represent topics in primary school natural sciences, such as food webs, life cycles, moon phases and human physiology. The corpus is based on the Allen Institute for Artificial Intelligence Diagrams (AI2D) dataset, a collection of diagrams with crowd-sourced descriptions, which was originally developed to support research on automatic diagram understanding and visual question answering. Building on the segmentation of diagram layouts in AI2D, the AI2D-RST corpus presents a new multi-layer annotation schema that provides a rich description of their multimodal structure. Annotated by trained experts, the layers describe (1) the grouping of diagram elements into perceptual units, (2) the connections set up by diagrammatic elements such as arrows and lines, and (3) the discourse relations between diagram elements, which are described using Rhetorical Structure Theory (RST). Each annotation layer in AI2D-RST is represented using a graph. The corpus is freely available for research and teaching.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 2; 2024: 12; 2025: 28; 2026: 15.

60 papers.

Sort:

By area:

  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  2. 2025VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interactioncited by 191
  3. 2024VITA: Towards Open-Source Interactive Omni Multimodal LLMcited by 172
  4. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  5. 2024Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Wantcited by 98
  6. 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 75
  7. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learningcited by 71
  8. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 67
  9. 2025Scaling Language-Free Visual Representation Learningcited by 62
  10. 2025Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Modelscited by 62
  11. 2024Reconstructive Visual Instruction Tuningcited by 54
  12. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoderscited by 49
38 more
  1. 2025Boosting MLLM Reasoning with Text-Debiased Hint-GRPOcited by 32
  2. 2026Beyond Language Modeling: An Exploration of Multimodal Pretrainingcited by 32
  3. 2024Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLMcited by 31
  4. 2024MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QAcited by 24
  5. 2025Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-trainingcited by 23
  6. 2023GlitchBench: Can large multimodal models detect video game glitches?cited by 21
  7. 2023Lyrics: Boosting Fine-grained Language-Vision Alignment and Comprehension via Semantic-aware Visual Objectscited by 17
  8. 2025MMPB: It's Time for Multi-Modal Personalizationcited by 14
  9. 2025PPTBench: Towards Holistic Evaluation of Large Language Models for PowerPoint Layout and Design Understandingcited by 11
  10. 2025Gatekeeper: Improving Model Cascades Through Confidence Tuningcited by 10
  11. 2021COSMic: A Coherence-Aware Generation Metric for Image Descriptionscited by 9
  12. 2024AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language Modelscited by 8
  13. 2024TG-LLaVA: Text Guided LLaVA via Learnable Latent Embeddingscited by 8
  14. 2025Multi-modal Summarization in Model-Based Engineering: Automotive Software Development Case Studycited by 6
  15. 2026Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understandingcited by 6
  16. 2024Ranked from Within: Ranking Large Multimodal Models Without Labelscited by 5
  17. 2025Beyond Perception: Evaluating Abstract Visual Reasoning through Multi-Stage Taskcited by 5
  18. 2025Investigating Redundancy in Multimodal Large Language Models with Multiple Vision Encoderscited by 5
  19. 2025D3ToM: Decider-Guided Dynamic Token Merging for Accelerating Diffusion MLLMscited by 5
  20. 2026WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Modelscited by 5
  21. 2024Detect, Describe, Discriminate: Moving Beyond VQA for MLLM Evaluationcited by 4
  22. 2025InsightVision: A Comprehensive, Multi-Level Chinese-based Benchmark for Evaluating Implicit Visual Semantics in Large Vision Language Modelscited by 4
  23. 2025CoMemo: LVLMs Need Image Context with Image Memorycited by 4
  24. 2025Astrea: A MOE-based Visual Understanding Model with Progressive Alignmentcited by 3
  25. 2025Vision-Centric Activation and Coordination for Multimodal Large Language Modelscited by 3
  26. 2025RL makes MLLMs see better than SFTcited by 3
  27. 2025Attention Guided Alignment in Efficient Vision-Language Modelscited by 3
  28. 2024EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignmentcited by 2
  29. 2025When Less is Enough: Adaptive Token Reduction for Efficient Image Representationcited by 2
  30. 2025Rethinking Visual Information Processing in Multimodal LLMscited by 2
  31. 2026OmniMapBench: Benchmarking Visual-Centric Reasoning on Diverse Map Documentscited by 2
  32. 2021Semiotically-grounded distant viewing of diagrams: insights from two multimodal corporacited by 1
  33. 2025Argus Inspection: Do Multimodal Large Language Models Possess the Eye of Panoptes?cited by 1
  34. 2025Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLMcited by 1
  35. 2026Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipescited by 1
  36. 2022ChartParser: Automatic Chart Parsing for Print-Impairedcited by 0
  37. 2025MGAA: Multi-Granular Adaptive Allocation fof Low-Rank Compression of LLMscited by 0
  38. 2025Towards Open World Detection: A Surveycited by 0

50 most cited of 60 All 60 (JSON)