# The Law of Large Documents: Understanding the Structure of Legal Contracts Using Visual Cues

arXiv: https://arxiv.org/abs/2107.08128 · HTML: https://mlsanity.com/paper/2107.08128/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 16 Jul 2021.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Large-Scale Multi-Label Text Classification on EU Legislation](https://mlsanity.com/paper/1906.02192/) (2019)
- [Chargrid: Towards Understanding 2D Documents](https://mlsanity.com/paper/1809.08799/) (2018)
- [A Neural Network Approach to Context-Sensitive Generation of Conversational Responses](https://mlsanity.com/paper/1506.06714/) (2015)
- [Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension](https://mlsanity.com/paper/2005.08056/) (2020)
- [Hierarchical Question Answering for Long Documents](https://mlsanity.com/paper/1611.01839/) (2016)
- [BERT Goes to Law School: Quantifying the Competitive Advantage of Access to Large Legal Corpora in Contract Understanding](https://mlsanity.com/paper/1911.00473/) (2019)
- [Long-length Legal Document Classification](https://mlsanity.com/paper/1912.06905/) (2019)
- [OCR Graph Features for Manipulation Detection in Documents](https://mlsanity.com/paper/2009.05158/) (2020)

## Built on by (most cited first, 4 of 4)

- [Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset](https://mlsanity.com/paper/2207.00220/) (2022)
- [ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extraction](https://mlsanity.com/paper/2210.08697/) (2022)
- [The Hidden Structure -- Improving Legal Document Understanding Through Explicit Text Formatting](https://mlsanity.com/paper/2505.12837/) (2025)
- [DeeperDive: The Unreasonable Effectiveness of Weak Supervision in Document Understanding A Case Study in Collaboration with UiPath Inc](https://mlsanity.com/paper/2208.08000/) (2022)

## Abstract (arXiv)

Large, pre-trained transformer models like BERT have achieved state-of-the-art results on document understanding tasks, but most implementations can only consider 512 tokens at a time. For many real-world applications, documents can be much longer, and the segmentation strategies typically used on longer documents miss out on document structure and contextual information, hurting their results on downstream tasks. In our work on legal agreements, we find that visual cues such as layout, style, and placement of text in a document are strong features that are crucial to achieving an acceptable level of accuracy on long documents. We measure the impact of incorporating such visual cues, obtained via computer vision methods, on the accuracy of document understanding tasks including document segmentation, entity extraction, and attribute classification. Our method of segmenting documents based on structural metadata out-performs existing methods on four long-document understanding tasks as measured on the Contract Understanding Atticus Dataset.