# Restoring Ancient Ideograph: A Multimodal Multitask Neural Network Approach

arXiv: https://arxiv.org/abs/2403.06682 · HTML: https://mlsanity.com/paper/2403.06682/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Enabling Language Models to Fill in the Blanks](https://mlsanity.com/paper/2005.05339/) (2020)
- [DE-GAN: A Conditional Generative Adversarial Network for Document Enhancement](https://mlsanity.com/paper/2010.08764/) (2020)
- [Blank Language Models](https://mlsanity.com/paper/2002.03079/) (2020)
- [Restoring ancient text using deep learning: a case study on Greek epigraphy](https://mlsanity.com/paper/1910.06262/) (2019)
- [Online and Offline Handwritten Chinese Character Recognition: A Comprehensive Study and New Benchmark](https://mlsanity.com/paper/1606.05763/) (2016)
- [Restoration of Fragmentary Babylonian Texts Using Recurrent Neural Networks](https://mlsanity.com/paper/2003.01912/) (2020)
- [Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation](https://mlsanity.com/paper/2104.05964/) (2021)
- [Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach](https://mlsanity.com/paper/2109.04513/) (2021)

## Built on by (most cited first, 1 of 1)

- [An Annotated Dataset of Errors in Premodern Greek and Baselines for Detecting Them](https://mlsanity.com/paper/2410.11071/) (2024)

## Abstract (arXiv)

Cultural heritage serves as the enduring record of human thought and history. Despite significant efforts dedicated to the preservation of cultural relics, many ancient artefacts have been ravaged irreversibly by natural deterioration and human actions. Deep learning technology has emerged as a valuable tool for restoring various kinds of cultural heritages, including ancient text restoration. Previous research has approached ancient text restoration from either visual or textual perspectives, often overlooking the potential of synergizing multimodal information. This paper proposes a novel Multimodal Multitask Restoring Model (MMRM) to restore ancient texts, particularly emphasising the ideograph. This model combines context understanding with residual visual information from damaged ancient artefacts, enabling it to predict damaged characters and generate restored images simultaneously. We tested the MMRM model through experiments conducted on both simulated datasets and authentic ancient inscriptions. The results show that the proposed method gives insightful restoration suggestions in both simulation experiments and real-world scenarios. To the best of our knowledge, this work represents the pioneering application of multimodal deep learning in ancient text restoration, which will contribute to the understanding of ancient society and culture in digital humanities fields.