# Cognate Transformer for Automated Phonological Reconstruction and Cognate Reflex Prediction

arXiv: https://arxiv.org/abs/2310.07487 · HTML: https://mlsanity.com/paper/2310.07487/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 11 Oct 2023.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Ab Antiquo: Neural Proto-language Reconstruction](https://mlsanity.com/paper/1908.02477/) (2019)
- [A New Framework for Fast Automated Phonological Reconstruction Using Trimmed Alignments and Sound Correspondence Patterns](https://mlsanity.com/paper/2204.04619/) (2022)
- [Multi-Source Neural Machine Translation with Missing Data](https://mlsanity.com/paper/1806.02525/) (2018)
- [Computational Historical Linguistics](https://mlsanity.com/paper/1805.08099/) (2018)

## Built on by (most cited first, 4 of 4)

- [Automated Cognate Detection as a Supervised Link Prediction Task with Cognate Transformer](https://mlsanity.com/paper/2402.02926/) (2024)
- [Semisupervised Neural Proto-Language Reconstruction](https://mlsanity.com/paper/2406.05930/) (2024)
- [A Likelihood Ratio Test of Genetic Relationship among Languages](https://mlsanity.com/paper/2404.00284/) (2024)
- [When Similar Means Different: Evaluating LLMs on Arabic--Hebrew Cognates](https://mlsanity.com/paper/2606.13218/) (2026)

## Abstract (arXiv)

Phonological reconstruction is one of the central problems in historical linguistics where a proto-word of an ancestral language is determined from the observed cognate words of daughter languages. Computational approaches to historical linguistics attempt to automate the task by learning models on available linguistic data. Several ideas and techniques drawn from computational biology have been successfully applied in the area of computational historical linguistics. Following these lines, we adapt MSA Transformer, a protein language model, to the problem of automated phonological reconstruction. MSA Transformer trains on multiple sequence alignments as input and is, thus, apt for application on aligned cognate words. We, hence, name our model as Cognate Transformer. We also apply the model on another associated task, namely, cognate reflex prediction, where a reflex word in a daughter language is predicted based on cognate words from other daughter languages. We show that our model outperforms the existing models on both tasks, especially when it is pre-trained on masked word prediction task.