# LIGHT: Multi-Modal Text Linking on Historical Maps

arXiv: https://arxiv.org/abs/2506.22589 · HTML: https://mlsanity.com/paper/2506.22589/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [DocFormer: End-to-End Transformer for Document Understanding](https://mlsanity.com/paper/2106.11539/) (2021)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [BERT: A Review of Applications in Natural Language Processing and Understanding](https://mlsanity.com/paper/2103.11943/) (2021)
- [Towards End-to-End Unified Scene Text Detection and Layout Analysis](https://mlsanity.com/paper/2203.15143/) (2022)
- [DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting](https://mlsanity.com/paper/2211.10772/) (2022)
- [Text Spotting Transformers](https://mlsanity.com/paper/2204.01918/) (2022)
- [Spatial Dependency Parsing for Semi-Structured Document Information Extraction](https://mlsanity.com/paper/2005.00642/) (2020)
- [GeoLayoutLM: Geometric Pre-training for Visual Information Extraction](https://mlsanity.com/paper/2304.10759/) (2023)
- [ICDAR 2023 Competition on Hierarchical Text Detection and Recognition](https://mlsanity.com/paper/2305.09750/) (2023)
- [Doc-GCN: Heterogeneous Graph Convolutional Networks for Document Layout Analysis](https://mlsanity.com/paper/2208.10970/) (2022)
- [Post-OCR Paragraph Recognition by Graph Convolutional Networks](https://mlsanity.com/paper/2101.12741/) (2021)
- [Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysis](https://mlsanity.com/paper/2401.11874/) (2024)
- [Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding](https://mlsanity.com/paper/2409.19672/) (2024)
- [DLAFormer: An End-to-End Transformer For Document Layout Analysis](https://mlsanity.com/paper/2405.11757/) (2024)
- [The mapKurator System: A Complete Pipeline for Extracting and Linking Text from Historical Maps](https://mlsanity.com/paper/2306.17059/) (2023)
- [Hierarchical Text Spotter for Joint Text Spotting and Layout Analysis](https://mlsanity.com/paper/2310.17674/) (2023)
- [Hyper-Local Deformable Transformers for Text Spotting on Historical Maps](https://mlsanity.com/paper/2506.15010/) (2025)
- [An Automatic Approach for Generating Rich, Linked Geo-Metadata from Historical Map Images](https://mlsanity.com/paper/2112.01671/) (2021)
- [Contextual Text Block Detection towards Scene Text Understanding](https://mlsanity.com/paper/2207.12955/) (2022)
- [Text Grouping Adapter: Adapting Pre-trained Text Detector for Layout Analysis](https://mlsanity.com/paper/2405.07481/) (2024)
- [Dynamic Relation Transformer for Contextual Text Block Detection](https://mlsanity.com/paper/2401.09232/) (2024)
- [Automatic Search of Multiword Place Names on Historical Maps](https://mlsanity.com/paper/2410.15586/) (2024)

## Abstract (arXiv)

Text on historical maps provides valuable information for studies in history, economics, geography, and other related fields. Unlike structured or semi-structured documents, text on maps varies significantly in orientation, reading order, shape, and placement. Many modern methods can detect and transcribe text regions, but they struggle to effectively ``link'' the recognized text fragments, e.g., determining a multi-word place name. Existing layout analysis methods model word relationships to improve text understanding in structured documents, but they primarily rely on linguistic features and neglect geometric information, which is essential for handling map text. To address these challenges, we propose LIGHT, a novel multi-modal approach that integrates linguistic, image, and geometric features for linking text on historical maps. In particular, LIGHT includes a geometry-aware embedding module that encodes the polygonal coordinates of text regions to capture polygon shapes and their relative spatial positions on an image. LIGHT unifies this geometric information with the visual and linguistic token embeddings from LayoutLMv3, a pretrained layout analysis model. LIGHT uses the cross-modal information to predict the reading-order successor of each text instance directly with a bi-directional learning strategy that enhances sequence robustness. Experimental results show that LIGHT outperforms existing methods on the ICDAR 2024/2025 MapText Competition data, demonstrating the effectiveness of multi-modal learning for historical map text linking.