# Unifying Scientific Communication: Fine-Grained Correspondence Across Scientific Media

arXiv: https://arxiv.org/abs/2605.05831 · HTML: https://mlsanity.com/paper/2605.05831/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Multi-modal Transformer for Video Retrieval](https://mlsanity.com/paper/2007.10639/) (2020)
- [Self-Supervised MultiModal Versatile Networks](https://mlsanity.com/paper/2006.16228/) (2020)
- [PP-OCR: A Practical Ultra Lightweight OCR System](https://mlsanity.com/paper/2009.09941/) (2020)
- [E5-V: Universal Embeddings with Multimodal Large Language Models](https://mlsanity.com/paper/2407.12580/) (2024)
- [DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents](https://mlsanity.com/paper/2101.11796/) (2021)
- [D2S: Document-to-Slide Generation Via Query-Based Text Summarization](https://mlsanity.com/paper/2105.03664/) (2021)
- [PP-DocLayout: A Unified Document Layout Detection Model to Accelerate Large-Scale Data Construction](https://mlsanity.com/paper/2503.17213/) (2025)
- [Paper2Video: Automatic Video Generation from Scientific Papers](https://mlsanity.com/paper/2510.05096/) (2025)
- [Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI Agents](https://mlsanity.com/paper/2509.06917/) (2025)
- [PresentAgent: Multimodal Agent for Presentation Video Generation](https://mlsanity.com/paper/2507.04036/) (2025)
- [P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark](https://mlsanity.com/paper/2505.17104/) (2025)
- [SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2401.09759/) (2024)
- [CS-PaperSum: A Large-Scale Dataset of AI-Generated Summaries for Scientific Papers](https://mlsanity.com/paper/2502.20582/) (2025)
- [MaViLS, a Benchmark Dataset for Video-to-Slide Alignment, Assessing Baseline Accuracy with a Multimodal Alignment Algorithm Leveraging Speech, OCR, and Visual Features](https://mlsanity.com/paper/2409.16765/) (2024)
- [DocVideoQA: Towards Comprehensive Understanding of Document-Centric Videos through Question Answering](https://mlsanity.com/paper/2503.15887/) (2025)
- [Attend to what I say: Highlighting relevant content on slides](https://mlsanity.com/paper/2601.10244/) (2026)

## Built on by (most cited first, 1 of 1)

- [AI for Auto-Research: Roadmap & User Guide](https://mlsanity.com/paper/2605.18661/) (2026)

## Abstract (arXiv)

The communication of scientific knowledge has become increasingly multimodal, spanning text, visuals, and speech through materials such as research papers, slides, and recorded presentations. These different representations collectively convey a study's reasoning, results, and insights, offering complementary perspectives that enrich understanding. However, despite their shared purpose, such materials are rarely connected in a structured way. The absence of explicit links across formats makes it difficult to trace how concepts, visuals, and explanations correspond, limiting unified exploration and analysis of research content. To address this gap, we introduce the Multimodal Conference Dataset (MCD), the first benchmark that integrates research papers, presentation videos, explanatory videos, and slides from the same works. We evaluate a range of embedding-based and vision-language models to assess their ability to discover fine-grained cross-format correspondences, establishing the first systematic benchmark for this task. Our results show that vision-language models are robust but struggle with fine-grained alignment, while embedding-based models capture text-visual correspondences well but equations and symbolic content form distinct clusters in the embedding space. These findings highlight both the strengths and limitations of current approaches and point to key directions for future research in multimodal scientific understanding. To ensure reproducibility, we release the resources for MCD at https://github.com/meghamariamkm2002/MCD