# Document Intelligence Metrics for Visually Rich Document Evaluation

arXiv: https://arxiv.org/abs/2205.11215 · HTML: https://mlsanity.com/paper/2205.11215/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 10 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 4 of 4)

- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [Graph Convolution for Multimodal Information Extraction from Visually Rich Documents](https://mlsanity.com/paper/1903.11279/) (2019)
- [DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extraction](https://mlsanity.com/paper/2103.05908/) (2021)

## Abstract (arXiv)

The processing of Visually-Rich Documents (VRDs) is highly important in information extraction tasks associated with Document Intelligence. We introduce DI-Metrics, a Python library devoted to VRD model evaluation comprising text-based, geometric-based and hierarchical metrics for information extraction tasks. We apply DI-Metrics to evaluate information extraction performance using publicly available CORD dataset, comparing performance of three SOTA models and one industry model. The open-source library is available on GitHub.