# Document Domain Randomization for Deep Learning Document Layout Extraction

arXiv: https://arxiv.org/abs/2105.14931 · HTML: https://mlsanity.com/paper/2105.14931/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 43 references that are in the map.
- First posted on arXiv 20 May 2021.

## Builds on (most cited first, 18 of 18)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Deep Learning is Robust to Massive Label Noise](https://mlsanity.com/paper/1705.10694/) (2017)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents](https://mlsanity.com/paper/2004.12629/) (2020)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [Extracting Scientific Figures with Distantly Supervised Neural Networks](https://mlsanity.com/paper/1804.02445/) (2018)
- [3D Simulation for Robot Arm Control with Deep Q-Learning](https://mlsanity.com/paper/1609.03759/) (2016)
- [Beagle: Automated Extraction and Interpretation of Visualizations from the Web](https://mlsanity.com/paper/1711.05962/) (2017)
- [VIS30K: A Collection of Figures and Tables from IEEE Visualization Conference Publications](https://mlsanity.com/paper/2101.01036/) (2020)
- [Toward A Deep Understanding of What Makes a Scientific Visualization Memorable](https://mlsanity.com/paper/1808.00607/) (2018)

## Built on by (most cited first, 1 of 1)

- [SynthDoc: Bilingual Documents Synthesis for Visual Document Understanding](https://mlsanity.com/paper/2408.14764/) (2024)

## Abstract (arXiv)

We present document domain randomization (DDR), the first successful transfer of convolutional neural networks (CNNs) trained only on graphically rendered pseudo-paper pages to real-world document segmentation. DDR renders pseudo-document pages by modeling randomized textual and non-textual contents of interest, with user-defined layout and font styles to support joint learning of fine-grained classes. We demonstrate competitive results using our DDR approach to extract nine document classes from the benchmark CS-150 and papers published in two domains, namely annual meetings of Association for Computational Linguistics (ACL) and IEEE Visualization (VIS). We compare DDR to conditions of style mismatch, fewer or more noisy samples that are more easily obtained in the real world. We show that high-fidelity semantic information is not necessary to label semantic classes but style mismatch between train and test can lower model accuracy. Using smaller training samples had a slightly detrimental effect. Finally, network models still achieved high test accuracy when correct labels are diluted towards confusing labels; this behavior hold across several classes.