# Leveraging Collection-Wide Similarities for Unsupervised Document Structure Extraction

arXiv: https://arxiv.org/abs/2402.13906 · HTML: https://mlsanity.com/paper/2402.13906/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 11 of 11)

- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [SECTOR: A Neural Model for Coherent Topic Segmentation and Classification](https://mlsanity.com/paper/1902.04793/) (2019)
- [Improving Unsupervised Dialogue Topic Segmentation with Utterance-Pair Coherence Scoring](https://mlsanity.com/paper/2106.06719/) (2021)
- [AlephBERT:A Hebrew Large Pre-Trained Language Model to Start-off your Hebrew NLP Application With](https://mlsanity.com/paper/2104.04052/) (2021)
- [Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications](https://mlsanity.com/paper/2212.09676/) (2022)
- [An Automatic Approach for Document-level Topic Model Evaluation](https://mlsanity.com/paper/1706.05140/) (2017)
- [Improving Topic Segmentation by Injecting Discourse Dependencies](https://mlsanity.com/paper/2209.08626/) (2022)
- [Processing the structure of documents: Logical Layout Analysis of historical newspapers in French](https://mlsanity.com/paper/2202.08125/) (2022)
- [Discourse Structure Extraction from Pre-Trained and Fine-Tuned Language Models in Dialogues](https://mlsanity.com/paper/2302.05895/) (2023)
- [Automated Extraction of Sentencing Decisions from Court Cases in the Hebrew Language](https://mlsanity.com/paper/2110.12383/) (2021)

## Built on by (most cited first, 2 of 2)

- [Language Modeling for the Future of Finance: A Survey into Metrics, Tasks, and Data Opportunities](https://mlsanity.com/paper/2504.07274/) (2025)
- [T5Score: A Methodology for Automatically Assessing the Quality of LLM Generated Multi-Document Topic Sets](https://mlsanity.com/paper/2407.17390/) (2024)

## Abstract (arXiv)

Document collections of various domains, e.g., legal, medical, or financial, often share some underlying collection-wide structure, which captures information that can aid both human users and structure-aware models. We propose to identify the typical structure of document within a collection, which requires to capture recurring topics across the collection, while abstracting over arbitrary header paraphrases, and ground each topic to respective document locations. These requirements pose several challenges: headers that mark recurring topics frequently differ in phrasing, certain section headers are unique to individual documents and do not reflect the typical structure, and the order of topics can vary between documents. Subsequently, we develop an unsupervised graph-based method which leverages both inter- and intra-document similarities, to extract the underlying collection-wide structure. Our evaluations on three diverse domains in both English and Hebrew indicate that our method extracts meaningful collection-wide structure, and we hope that future work will leverage our method for multi-document applications and structure-aware models.