# Multi-Modal Semantic Parsing for the Interpretation of Tombstone Inscriptions

arXiv: https://arxiv.org/abs/2507.04377 · HTML: https://mlsanity.com/paper/2507.04377/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 14 of its 56 references that are in the map.
- First posted on arXiv 6 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 14 of 14)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones?](https://mlsanity.com/paper/2412.01175/) (2024)
- [Neural Semantic Parsing by Character-based Translation: Experiments with Abstract Meaning Representations](https://mlsanity.com/paper/1705.09980/) (2017)
- [Exploring Neural Methods for Parsing Discourse Representation Structures](https://mlsanity.com/paper/1810.12579/) (2018)
- ["You Are An Expert Linguistic Annotator": Limits of LLMs as Analyzers of Abstract Meaning Representation](https://mlsanity.com/paper/2310.17793/) (2023)
- [RIGA at SemEval-2016 Task 8: Impact of Smatch Extensions and Character-Level Neural Translation on AMR Parsing Accuracy](https://mlsanity.com/paper/1604.01278/) (2016)
- [Neural Semantic Parsing with Extremely Rich Symbolic Meaning Representations](https://mlsanity.com/paper/2404.12698/) (2024)
- [PMB5: Gaining More Insight into Neural Semantic Parsing with Challenging Benchmarks](https://mlsanity.com/paper/2404.08354/) (2024)
- [Visual Semantic Parsing: From Images to Abstract Meaning Representation](https://mlsanity.com/paper/2210.14862/) (2022)
- [Discourse Representation Structure Parsing for Chinese](https://mlsanity.com/paper/2306.09725/) (2023)
- [MACT: Model-Agnostic Cross-Lingual Training for Discourse Representation Structure Parsing](https://mlsanity.com/paper/2406.01052/) (2024)
- [Attention Guided Semantic Relationship Parsing for Visual Question Answering](https://mlsanity.com/paper/2010.01725/) (2020)
- [Character-level Representations Improve DRS-based Semantic Parsing Even in the Age of BERT](https://mlsanity.com/paper/2011.04308/) (2020)
- [Scope-enhanced Compositional Semantic Parsing for DRT](https://mlsanity.com/paper/2407.01899/) (2024)

## Abstract (arXiv)

Tombstones are historically and culturally rich artifacts, encapsulating individual lives, community memory, historical narratives and artistic expression. Yet, many tombstones today face significant preservation challenges, including physical erosion, vandalism, environmental degradation, and political shifts. In this paper, we introduce a novel multi-modal framework for tombstones digitization, aiming to improve the interpretation, organization and retrieval of tombstone content. Our approach leverages vision-language models (VLMs) to translate tombstone images into structured Tombstone Meaning Representations (TMRs), capturing both image and text information. To further enrich semantic parsing, we incorporate retrieval-augmented generation (RAG) for integrate externally dependent elements such as toponyms, occupation codes, and ontological concepts. Compared to traditional OCR-based pipelines, our method improves parsing accuracy from an F1 score of 36.1 to 89.5. We additionally evaluate the model's robustness across diverse linguistic and cultural inscriptions, and simulate physical degradation through image fusion to assess performance under noisy or damaged conditions. Our work represents the first attempt to formalize tombstone understanding using large vision-language models, presenting implications for heritage preservation.