# Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers

arXiv: https://arxiv.org/abs/2002.06144 · HTML: https://mlsanity.com/paper/2002.06144/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 10 of 10)

- [Vision Grid Transformer for Document Layout Analysis](https://mlsanity.com/paper/2308.14978/) (2023)
- [VSR: A Unified Framework for Document Layout Analysis combining Vision, Semantics and Relations](https://mlsanity.com/paper/2105.06220/) (2021)
- [ViBERTgrid: A Jointly Trained Multi-Modal 2D Document Representation for Key Information Extraction from Documents](https://mlsanity.com/paper/2105.11672/) (2021)
- [DocBed: A Multi-Stage OCR Solution for Documents with Complex Layouts](https://mlsanity.com/paper/2202.01414/) (2022)
- [Page Layout Analysis of Text-heavy Historical Documents: a Comparison of Textual and Visual Approaches](https://mlsanity.com/paper/2212.13924/) (2022)
- [Processing the structure of documents: Logical Layout Analysis of historical newspapers in French](https://mlsanity.com/paper/2202.08125/) (2022)
- [DocAligner: Annotating Real-world Photographic Document Images by Simply Taking Pictures](https://mlsanity.com/paper/2306.05749/) (2023)
- [Palmira: A Deep Deformable Network for Instance Segmentation of Dense and Uneven Layouts in Handwritten Manuscripts](https://mlsanity.com/paper/2108.09436/) (2021)
- [DREAM: Document Reconstruction via End-to-end Autoregressive Model](https://mlsanity.com/paper/2507.05805/) (2025)
- [Including Keyword Position in Image-based Models for Act Segmentation of Historical Registers](https://mlsanity.com/paper/2109.08477/) (2021)

## Abstract (arXiv)

The massive amounts of digitized historical documents acquired over the last decades naturally lend themselves to automatic processing and exploration. Research work seeking to automatically process facsimiles and extract information thereby are multiplying with, as a first essential step, document layout analysis. If the identification and categorization of segments of interest in document images have seen significant progress over the last years thanks to deep learning techniques, many challenges remain with, among others, the use of finer-grained segmentation typologies and the consideration of complex, heterogeneous documents such as historical newspapers. Besides, most approaches consider visual features only, ignoring textual signal. In this context, we introduce a multimodal approach for the semantic segmentation of historical newspapers that combines visual and textual features. Based on a series of experiments on diachronic Swiss and Luxembourgish newspapers, we investigate, among others, the predictive power of visual and textual features and their capacity to generalize across time and sources. Results show consistent improvement of multimodal models in comparison to a strong visual baseline, as well as better robustness to high material variance.