# From Rosetta to Match-Up: A Paired Corpus of Linguistic Puzzles with Human and LLM Benchmarks

arXiv: https://arxiv.org/abs/2605.13408 · HTML: https://mlsanity.com/paper/2605.13408/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 5 of 5)

- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [PuzzLing Machines: A Challenge on Learning From Small Data](https://mlsanity.com/paper/2004.13161/) (2020)
- [modeLing: A Novel Dataset for Testing Linguistic Reasoning in Language Models](https://mlsanity.com/paper/2406.17038/) (2024)
- [LINGOLY-TOO: Disentangling Reasoning from Knowledge with Templatised Orthographic Obfuscation](https://mlsanity.com/paper/2503.02972/) (2025)
- [Can LLMs Generate and Solve Linguistic Olympiad Puzzles?](https://mlsanity.com/paper/2509.21820/) (2025)

## Abstract (arXiv)

In this paper, we examine linguistic puzzles used in high school linguistics competitions, focusing on two common formats: Rosetta Stone and Match-Up. We propose a systematic procedure for converting existing Rosetta Stone puzzles into corresponding Match-Up counterparts. Because linguistic puzzle creation is complex and time-consuming, our method provides an efficient way to accelerate the generation of new puzzles. We evaluate the resulting Rosetta Stone-Match-Up pairs with both human participants and large language models (LLMs). Our results show that both expert human solvers and LLMs display an all-or-nothing pattern on Match-Up puzzles, either solving them completely or failing entirely. This work contributes a new dataset of paired puzzles and provides a detailed evaluation of puzzle difficulty across formats, offering insights into both human and machine linguistic reasoning.