# Language Models are Crossword Solvers

arXiv: https://arxiv.org/abs/2406.09043 · HTML: https://mlsanity.com/paper/2406.09043/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Decrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLP](https://mlsanity.com/paper/2104.08620/) (2021)
- [Cryptonite: A Cryptic Crossword Benchmark for Extreme Ambiguity in Language](https://mlsanity.com/paper/2103.01242/) (2021)
- [Automated Crossword Solving](https://mlsanity.com/paper/2205.09665/) (2022)
- [Dr.Fill: Crosswords and an Implemented Solver for Singly Weighted CSPs](https://mlsanity.com/paper/1401.4597/) (2014)
- [Large Language Models as Minecraft Agents](https://mlsanity.com/paper/2402.08392/) (2024)
- [Down and Across: Introducing Crossword-Solving as a New NLP Benchmark](https://mlsanity.com/paper/2205.10442/) (2022)

## Built on by (most cited first, 5 of 5)

- [CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs with Controllable Puzzle Generation](https://mlsanity.com/paper/2504.00043/) (2025)
- [TextGames: Learning to Self-Play Text-Based Puzzle Games via Language Model Reasoning](https://mlsanity.com/paper/2502.18431/) (2025)
- [Beyond In-Distribution Success: Scaling Curves of CoT Granularity for Language Model Generalization](https://mlsanity.com/paper/2502.18273/) (2025)
- [A Reasoning-Based Approach to Cryptic Crossword Clue Solving](https://mlsanity.com/paper/2506.04824/) (2025)
- [Orthographic Constraint Satisfaction and Human Difficulty Alignment in Large Language Models](https://mlsanity.com/paper/2511.21086/) (2025)

## Abstract (arXiv)

Crosswords are a form of word puzzle that require a solver to demonstrate a high degree of proficiency in natural language understanding, wordplay, reasoning, and world knowledge, along with adherence to character and length constraints. In this paper we tackle the challenge of solving crosswords with large language models (LLMs). We demonstrate that the current generation of language models shows significant competence at deciphering cryptic crossword clues and outperforms previously reported state-of-the-art (SoTA) results by a factor of 2-3 in relevant benchmarks. We also develop a search algorithm that builds off this performance to tackle the problem of solving full crossword grids with out-of-the-box LLMs for the very first time, achieving an accuracy of 93% on New York Times crossword puzzles. Additionally, we demonstrate that LLMs generalize well and are capable of supporting answers with sound rationale.