Arctic-TILT. Business Document Understanding at Sub-Billion Scale
Not on the map. Why?
Abstract (arXiv)
The vast portion of workloads employing LLMs involves answering questions grounded on PDF or scan content. We introduce the Arctic-TILT achieving accuracy on par with models 1000× its size on these use cases. It can be fine-tuned and deployed on a single 24GB GPU, lowering operational costs while processing Visually Rich Documents with up to 400k tokens. The model establishes state-of-the-art results on seven diverse Document Understanding benchmarks, as well as provides reliable confidence scores and quick inference, which are essential for processing files in large-scale or time-sensitive enterprise environments.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 52 of its 75 references that are in the map. First posted on arXiv 8 Aug 2024.
Builds on
52 of 75 references.
References · 52
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
- 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
- 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
- 2024PaliGemma: A versatile 3B VLM for transfercited by 679
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
38 more
- 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 406
- 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 360
- 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
- 2019Scene Text Visual Question Answeringcited by 327
- 2019Character Region Awareness for Text Detectioncited by 273
- 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 272
- 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
- 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
- 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
- 2021Self-attention Does Not Need O(n2) Memorycited by 152
- 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 149
- 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
- 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
- 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
- 2022Hierarchical multimodal transformers for Multi-Page DocVQAcited by 102
- 2019Graph Convolution for Multimodal Information Extraction from Visually Rich Documentscited by 88
- 2021Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layoutscited by 82
- 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
- 2022Efficient Long-Text Understanding with Short-Text Modelscited by 71
- 2020PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networkscited by 70
- 2022ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understandingcited by 67
- 2024Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classificationcited by 66
- 2020Spatial Dependency Parsing for Semi-Structured Document Information Extractioncited by 65
- 2021Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solutioncited by 64
- 2024LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Reportcited by 47
- 2024DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimmingcited by 35
- 2019Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solvingcited by 34
- 2024Tiny Titans: Can Smaller Large Language Models Punch Above Their Weight in the Real World for Meeting Summarization?cited by 32
- 2022FiDO: Fusion-in-Decoder optimized for stronger performance and faster inferencecited by 31
- 2024Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Modelscited by 30
- 2024LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understandingcited by 24
- 2024GRAM: Global Reasoning for Multi-Page VQAcited by 23
- 2022Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understandingcited by 20
- 2019A Neural Model for Text Localization, Transcription and Named Entity Recognition in Full Pagescited by 16
- 2024Visually Guided Generative Text-Layout Pre-training for Document Intelligencecited by 12
- 2024StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyondcited by 11
- 2020Sparsifying Transformer Models with Trainable Representation Poolingcited by 9
- 2024Notes on Applicability of GPT-4 to Document Understandingcited by 7
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.