# Can Large Language Models Code Like a Linguist?: A Case Study in Low Resource Sound Law Induction

arXiv: https://arxiv.org/abs/2406.12725 · HTML: https://mlsanity.com/paper/2406.12725/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 53 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 22 of 22)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [Magicoder: Empowering Code Generation with OSS-Instruct](https://mlsanity.com/paper/2312.02120/) (2023)
- [Jigsaw: Large Language Models meet Program Synthesis](https://mlsanity.com/paper/2112.02969/) (2021)
- [Interactive Code Generation via Test-Driven User-Intent Formalization](https://mlsanity.com/paper/2208.05950/) (2022)
- [Is Programming by Example solved by LLMs?](https://mlsanity.com/paper/2406.08316/) (2024)
- [Ab Antiquo: Neural Proto-language Reconstruction](https://mlsanity.com/paper/1908.02477/) (2019)
- [ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis](https://mlsanity.com/paper/2307.13883/) (2023)
- [Optimal Neural Program Synthesis from Multimodal Specifications](https://mlsanity.com/paper/2010.01678/) (2020)
- [Constructions Are So Difficult That Even Large Language Models Get Them Right for the Wrong Reasons](https://mlsanity.com/paper/2403.17760/) (2024)
- [Improved Neural Protoform Reconstruction via Reflex Prediction](https://mlsanity.com/paper/2403.18769/) (2024)
- [Verbing Weirds Language (Models): Evaluation of English Zero-Derivation in Five LLMs](https://mlsanity.com/paper/2403.17856/) (2024)
- [Sample-efficient Linguistic Generalizations through Program Synthesis: Experiments with Phonology Problems](https://mlsanity.com/paper/2106.06566/) (2021)
- [Automating Sound Change Prediction for Phylogenetic Inference: A Tukanoan Case Study](https://mlsanity.com/paper/2402.01582/) (2024)

## Built on by (most cited first, 2 of 2)

- [LingBench++: A Linguistically-Informed Benchmark and Reasoning Framework for Multi-Step and Cross-Cultural Inference with LLMs](https://mlsanity.com/paper/2507.16809/) (2025)
- [PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics](https://mlsanity.com/paper/2505.23126/) (2025)

## Abstract (arXiv)

Historical linguists have long written a kind of incompletely formalized ''program'' that converts reconstructed words in an ancestor language into words in one of its attested descendants that consist of a series of ordered string rewrite functions (called sound laws). They do this by observing pairs of words in the reconstructed language (protoforms) and the descendent language (reflexes) and constructing a program that transforms protoforms into reflexes. However, writing these programs is error-prone and time-consuming. Prior work has successfully scaffolded this process computationally, but fewer researchers have tackled Sound Law Induction (SLI), which we approach in this paper by casting it as Programming by Examples. We propose a language-agnostic solution that utilizes the programming ability of Large Language Models (LLMs) by generating Python sound law programs from sound change examples. We evaluate the effectiveness of our approach for various LLMs, propose effective methods to generate additional language-agnostic synthetic data to fine-tune LLMs for SLI, and compare our method with existing automated SLI methods showing that while LLMs lag behind them they can complement some of their weaknesses.