mlsanityData to 14 Oct 2026

Apollo Restore: A Foundation LLM for Historical Greek Optimized for Fill-in-the-Middle Restoration of Ancient Greek Texts

arXiv 2609.22455 ↗First posted 18 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We present Apollo Restore, a 24-billion-parameter large language model for restoring lacunae---physical gaps---in fragmentary Ancient Greek texts. Fine-tuned from Mistral Small with a fill-in-the-middle objective, Apollo Restore reconstructs missing spans without requiring oracle knowledge of their length. To our knowledge, it is the first large-scale decoder model for historical Greek, and the first for any ancient Mediterranean language. Evaluated as in prior work, on short gaps of up to ten characters, Apollo Restore places the correct restoration among its top twenty candidates for 80.6%/54.6%/61.0% of documentary-papyrus, literary-papyrus, and stone-inscription lacunae, exceeding the strongest published models by 1.6×/2.6×/1.4×. Prior evaluation protocols, however, inflate scores through a bias toward trivially short gaps; under a length-balanced metric Apollo Restore's advantage over the strongest published models grows to 2.3×/3.5×/1.6× and degrades gracefully, even given incorrect length hints. In a blind study, 20 expert papyrologists, epigraphists, and philologists strongly preferred Apollo Restore to the strongest baseline and judged its performance at least as good as human restorations in 77% of cases. Apollo Restore also improves the published reading of PHerc. 1667---a papyrus roll carbonised in the eruption of Vesuvius in 79 CE and digitally unrolled and edited after Apollo Restore's training data was compiled. Apollo Restore is an output of the Decoding Antiquity initiative to build specialized LLMs for historical languages and manuscripts, led by the Austrian Academy of Sciences.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 36 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

18 of 36 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  9. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  10. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 498
  11. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  12. 2020Latin BERT: A Contextual Language Model for Classical Philologycited by 43
6 more
  1. 2019Restoring ancient text using deep learning: a case study on Greek epigraphycited by 25
  2. 2023Exploring Large Language Models for Classical Philologycited by 10
  3. 2025Institutional Books 1.0: A 242B token dataset from Harvard Library's collections, refined for accuracy and usabilitycited by 7
  4. 2024Instruct-Tuning Pretrained Causal Language Models for Ancient Greek Papyrology and Epigraphycited by 3
  5. 2024An Annotated Dataset of Errors in Premodern Greek and Baselines for Detecting Themcited by 1
  6. 2026Complete virtual unwrapping and reading of a rolled Herculaneum papyruscited by 1

Built on by

Not cited yet.