mlsanityData to 7 Oct 2026

The unreasonable effectiveness of pattern matching

arXiv 2601.11432 ↗First posted 16 Jan 2026cs.CLcited by 2

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

We report on an astonishing ability of large language models (LLMs) to make sense of "Jabberwocky" language in which most or all content words have been randomly replaced by nonsense strings, e.g., translating "He dwushed a ghanc zawk" to "He dragged a spare chair". This result addresses ongoing controversies regarding how to best think of what LLMs are doing: are they a language mimic, a database, a blurry version of the Web? The ability of LLMs to recover meaning from structural patterns speaks to the unreasonable effectiveness of pattern-matching. Pattern-matching is not an alternative to "real" intelligence, but rather a key ingredient.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 3 of its 42 references that are in the map. First posted on arXiv 16 Jan 2026.

Builds on

3 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023GPT-4 Can't Reason21 Jul

Other references · 2

  1. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  2. 2023Language Modeling Is Compressioncited by 225

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026The Astonishing Ability of Large Language Models to Parse Jabberwockified Languagecited by 0
  2. 2026Reasoning as Pattern Matching: Shared Mechanisms in Human and LLM Everyday Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UNVEILING: What Makes Linguistics Olympiad Puzzles Tricky for LLMs?
  2. 2023Large Linguistic Models: Investigating LLMs' metalinguistic abilities
  3. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
  4. 2023Evaluating the Deductive Competence of Large Language Models
  5. 2025Position: On the Methodological Pitfalls of Evaluating Base LLMs for Reasoning
  6. 2024Enhancing LLM Evaluations: The Garbling Trick
  7. 2026Do LLMs Make More Mistakes If They Do Not Believe the Input Data?
  8. 2024LLMs' Understanding of Natural Language Revealed