mlsanityData to 14 Oct 2026

Nougat: Neural Optical Understanding for Academic Documents

arXiv 2308.13418 ↗First posted 25 Aug 2023cs.LG cs.CVcited by 245

Not on the map. Why?

Cited by 50 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Scientific knowledge is predominantly stored in books and scientific journals, often in the form of PDFs. However, the PDF format leads to a loss of semantic information, particularly for mathematical expressions. We propose Nougat (Neural Optical Understanding for Academic Documents), a Visual Transformer model that performs an Optical Character Recognition (OCR) task for processing scientific documents into a markup language, and demonstrate the effectiveness of our model on a new dataset of scientific documents. The proposed approach offers a promising solution to enhance the accessibility of scientific knowledge in the digital age, by bridging the gap between human-readable documents and machine-readable text. We release the models and code to accelerate future work on scientific text recognition.

Key facts. Cited by 245 papers in the map (data through 14 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

25 of 49 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2018Hierarchical Neural Story Generationcited by 1,193
  7. 2022Galactica: A Large Language Model for Sciencecited by 744
  8. 2018Albumentations: fast and flexible image augmentationscited by 560
  9. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  10. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 528
  11. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  12. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
13 more
  1. 2021OCR-free Document Understanding Transformercited by 407
  2. 2021TrOCR: Transformer-based Optical Character Recognition with Pre-trained Modelscited by 279
  3. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  4. 2022Scene Text Recognition with Permuted Autoregressive Sequence Modelscited by 149
  5. 2019Cycle-Consistency for Robust Visual Question Answeringcited by 105
  6. 2016Image-to-Markup Generation with Coarse-to-Fine Attentioncited by 79
  7. 2022End-to-end Document Recognition and Understanding with Dessurtcited by 73
  8. 2021Rethinking Text Line Recognition Modelscited by 45
  9. 2018Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognitioncited by 36
  10. 2022OCR-IDL: OCR Annotations for Industry Document Library Datasetcited by 35
  11. 2021Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformercited by 34
  12. 2020ConvMath: A Convolutional Sequence Network for Mathematical Expression Recognitioncited by 9
  13. 2018Teaching Machines to Code: Neural Markup Generation with Visual Attentioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 60; 2025: 66; 2026: 111.

245 papers.

Sort:

By area:

  1. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  2. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  3. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
  4. 2024What matters when building vision-language models?cited by 326
  5. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  6. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  7. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  8. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  9. 2024Building and better understanding vision-language models: insights and future directionscited by 173
  10. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 152
  11. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 150
  12. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 125
38 more
  1. 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 120
  2. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 116
  3. 2024GUICourse: From General Vision Language Models to Versatile GUI Agentscited by 114
  4. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
  5. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 89
  6. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
  7. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
  8. 2024CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacycited by 81
  9. 2024ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuningcited by 79
  10. 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 79
  11. 2024Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilitiescited by 69
  12. 2024OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviewscited by 65
  13. 2024Focus Anywhere for Fine-grained Multi-page Document Understandingcited by 62
  14. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 53
  15. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 47
  16. 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extractioncited by 44
  17. 2025EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answeringcited by 44
  18. 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 42
  19. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
  20. 2024OneChart: Purify the Chart Structural Extraction via One Auxiliary Tokencited by 39
  21. 2023MechGPT, a language-based strategy for mechanics and materials modeling that connects knowledge across scales, disciplines and modalitiescited by 38
  22. 2025POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversioncited by 34
  23. 2024Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysiscited by 33
  24. 2024DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentationcited by 32
  25. 2024TextHawk: Exploring Efficient Fine-Grained Perception of Multimodal Large Language Modelscited by 27
  26. 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 27
  27. 2024OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generationcited by 26
  28. 2024What Can Natural Language Processing Do for Peer Review?cited by 25
  29. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 23
  30. 2026LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCRcited by 23
  31. 2023AI-TA: Towards an Intelligent Question-Answer Teaching Assistant using Open-Source LLMscited by 22
  32. 2026AgentOCR: Reimagining Agent History via Optical Self-Compressioncited by 22
  33. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Promptingcited by 22
  34. 2026Qianfan-OCR: A Unified End-to-End Model for Document Intelligencecited by 21
  35. 2025AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Paperscited by 20
  36. 2025HunyuanOCR Technical Reportcited by 20
  37. 2024MolParser: End-to-end Visual Recognition of Molecule Structures in the Wildcited by 17
  38. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsingcited by 17

50 most cited of 245 All 245 (JSON)