mlsanityData to 14 Oct 2026

Architecture-Agnostic Curriculum Learning for Document Understanding: Empirical Evidence from Text-Only and Multimodal

arXiv 2602.21225 ↗First posted 2 Feb 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We investigate whether progressive data scheduling -- a curriculum learning strategy that incrementally increases training data exposure (33%rightarrow67%rightarrow100%) -- yields consistent efficiency gains across architecturally distinct document understanding models. By evaluating BERT (text-only, 110M parameters) and LayoutLMv3 (multimodal, 126M parameters) on the FUNSD and CORD benchmarks, we establish that this schedule reduces wall-clock training time by approximately 33%, commensurate with the reduction from 6.67 to 10.0 effective epoch-equivalents of data. To isolate curriculum effects from compute reduction, we introduce matched-compute baselines (Standard-7) that control for total gradient updates. On the FUNSD dataset, the curriculum significantly outperforms the matched-compute baseline for BERT (DeltaF1 = +0.023, p=0.022, dz=3.83), constituting evidence for a genuine scheduling benefit in capacity-constrained models. In contrast, no analogous benefit is observed for LayoutLMv3 (p=0.621), whose multimodal representations provide sufficient inductive bias. On the CORD dataset, all conditions converge to equivalent F1 scores (≥0.947) irrespective of scheduling, indicating a performance ceiling. Schedule ablations comparing progressive, two-phase, reverse, and random pacing confirm that the efficiency gain derives from reduced data volume rather than ordering. Taken together, these findings demonstrate that progressive scheduling is a reliable compute-reduction strategy across model families, with curriculum-specific benefits contingent on the interaction between model capacity and task complexity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

21 of 31 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  3. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  4. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  5. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  6. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  7. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  8. 2021OCR-free Document Understanding Transformercited by 407
  9. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  10. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  11. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 286
  12. 2021Curriculum Learning: A Surveycited by 259
9 more
  1. 2017Automated Curriculum Learning for Neural Networkscited by 257
  2. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understandingcited by 168
  3. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  4. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 116
  5. 2020When Do Curricula Work?cited by 106
  6. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  7. 2023GeoLayoutLM: Geometric Pre-training for Visual Information Extractioncited by 40
  8. 2025Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learningcited by 26
  9. 2024EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Trainingcited by 18

Built on by

Not cited yet.