mlsanityData to 7 Oct 2026

Are LLMs Good Cryptic Crossword Solvers?

arXiv 2403.12094 ↗First posted 15 Mar 2024cs.AI cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Cryptic crosswords are puzzles that rely not only on general knowledge but also on the solver's ability to manipulate language on different levels and deal with various types of wordplay. Previous research suggests that solving such puzzles is a challenge even for modern NLP models. However, the abilities of large language models (LLMs) have not yet been tested on this task. In this paper, we establish the benchmark results for three popular LLMs -- LLaMA2, Mistral, and ChatGPT -- showing that their performance on this task is still far from that of humans.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 11 of its 26 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

11 of 26 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 204
  8. 2019Pun Generation with Surprisecited by 37
  9. 2019Pun-GAN: Generative Adversarial Network for Pun Generationcited by 18
  10. 2021Decrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLPcited by 16
  11. 2021Cryptonite: A Cryptic Crossword Benchmark for Extreme Ambiguity in Languagecited by 15

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 5.

7 papers.

  1. 2025CrossWordBench: Evaluating the Reasoning Capabilities of LLMs and LVLMs with Controllable Puzzle Generationcited by 9
  2. 2025PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzlescited by 5
  3. 2024Non Verbis, Sed Rebus: Large Language Models are Weak Solvers of Italian Rebusescited by 2
  4. 2025Do You Get the Hint? Benchmarking LLMs on the Board Game Conceptcited by 2
  5. 2025left|,circlearrowright,boxedBUS,right|: A Large and Diverse Multimodal Benchmark for evaluating the ability of Vision-Language Models to understand Rebus Puzzlescited by 1
  6. 2024What Makes Cryptic Crosswords Challenging for LLMs?cited by 0
  7. 2025A Reasoning-Based Approach to Cryptic Crossword Clue Solvingcited by 0