mlsanityData to 7 Oct 2026

End-to-end Document Recognition and Understanding with Dessurt

arXiv 2203.16618 ↗First posted 30 Mar 2022cs.CVcited by 73

Not on the map. Why?

Abstract (arXiv)

We introduce Dessurt, a relatively simple document understanding transformer capable of being fine-tuned on a greater variety of document tasks than prior methods. It receives a document image and task string as input and generates arbitrary text autoregressively as output. Because Dessurt is an end-to-end architecture that performs text recognition in addition to the document understanding, it does not require an external recognition model as prior methods do. Dessurt is a more flexible model than prior methods and is able to handle a variety of document domains and tasks. We show that this model is effective at 9 different dataset-task combinations.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 30 Mar 2022.

Builds on

25 of 40 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  8. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
  9. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
  10. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  11. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
  12. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 212
13 more
  1. 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
  2. 2021BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documentscited by 104
  3. 2020OrigamiNet: Weakly-Supervised, Segmentation-Free, One-Step, Full Page Text Recognition by learning to unfoldcited by 53
  4. 2020End-to-end Handwritten Paragraph Text Recognition Using a Vertical Attention Networkcited by 50
  5. 2016Joint Line Segmentation and Transcription for End-to-End Handwritten Paragraph Recognitioncited by 46
  6. 2020Text and Style Conditioned GAN for Generation of Offline Handwriting Linescited by 35
  7. 2019Deep Visual Template-Free Form Parsingcited by 24
  8. 2021Visual FUDGE: Form Understanding via Dynamic Graph Editingcited by 15
  9. 2021Cost-effective End-to-end Information Extraction for Semi-structured Document Imagescited by 14
  10. 2021Asking questions on handwritten document collectionscited by 14
  11. 2019A Computationally Efficient Pipeline Approach to Full Page Offline Handwritten Text Recognitioncited by 9
  12. 2021DocReader: Bounding-Box Free Training of a Document Information Extraction Modelcited by 4
  13. 2022Recognition-free Question Answering on Handwritten Document Collectionscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 20; 2024: 28; 2025: 13; 2026: 7.

73 papers.

Sort:

By area:

  1. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
  2. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 240
  3. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 173
  4. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understandingcited by 167
  5. 2023UReader: Universal OCR-free Visually-situated Language Understanding with Multimodal Large Language Modelcited by 143
  6. 2022MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derenderingcited by 132
  7. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 112
  8. 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 102
  9. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 95
  10. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understandingcited by 94
  11. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
  12. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
38 more
  1. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 51
  2. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  3. 2024Small Language Model Meets with Reinforced Vision Vocabularycited by 43
  4. 2023StrucTexTv2: Masked Visual-Textual Prediction for Document Image Pre-trainingcited by 39
  5. 2024HRVDA: High-Resolution Visual Document Assistantcited by 37
  6. 2024DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimmingcited by 35
  7. 2024Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentationcited by 30
  8. 2023ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rulescited by 24
  9. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 22
  10. 2024Deep Learning based Visually Rich Document Content Understanding: A Surveycited by 21
  11. 2023DocParser: End-to-end OCR-free Information Extraction from Visually Rich Documentscited by 19
  12. 2023Attention Where It Matters: Rethinking Visual Document Understanding with Selective Region Concentrationcited by 18
  13. 2024Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQAcited by 18
  14. 2022Evaluating Out-of-Distribution Performance on Document Image Classifierscited by 16
  15. 2024Improving Language Understanding from Screenshotscited by 14
  16. 2024Visually Guided Generative Text-Layout Pre-training for Document Intelligencecited by 12
  17. 2024Hierarchical Visual Feature Aggregation for OCR-Free Document Understandingcited by 12
  18. 2023Key-value information extraction from full handwritten pagescited by 9
  19. 2024LAPDoc: Layout-Aware Prompting for Documentscited by 9
  20. 2024DANIEL: A fast Document Attention Network for Information Extraction and Labelling of handwritten documentscited by 9
  21. 2024BuDDIE: A Business Document Dataset for Multi-task Information Extractioncited by 8
  22. 2024Multi-Page Document Visual Question Answering using Self-Attention Scoring Mechanismcited by 8
  23. 2024VisFocus: Prompt-Guided Vision Encoders for OCR-Free Dense Document Understandingcited by 8
  24. 2025BoundingDocs: a Unified Dataset for Document Question Answering with Spatial Annotationscited by 8
  25. 2023SelfDocSeg: A Self-Supervised vision-based Approach towards Document Segmentationcited by 7
  26. 2024μgat: Improving Single-Page Document Parsing by Providing Multi-Page Contextcited by 7
  27. 2023PixT3: Pixel-based Table-To-Text Generationcited by 6
  28. 2024Extracting Training Data from Document-Based VQA Modelscited by 6
  29. 2024Deep Learning based Key Information Extraction from Business Documents: Systematic Literature Reviewcited by 6
  30. 2022On Web-based Visual Corpus Construction for Visual Document Understandingcited by 5
  31. 2023On Evaluation of Document Classification using RVL-CDIPcited by 5
  32. 2023PHD: Pixel-Based Language Modeling of Historical Documentscited by 5
  33. 2025DocVXQA: Context-Aware Visual Explanations for Document Question Answeringcited by 5
  34. 2025Towards Natural Language-Based Document Image Retrieval: New Dataset and Benchmarkcited by 5
  35. 2023Visually-Situated Natural Language Understanding with Contrastive Reading Model and Frozen Large Language Modelscited by 4
  36. 2023FATURA: A Multi-Layout Invoice Image Dataset for Document Analysis and Understandingcited by 4
  37. 2024SynthDoc: Bilingual Documents Synthesis for Visual Document Understandingcited by 4
  38. 2025Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and Trendscited by 4

50 most cited of 73 All 73 (JSON)