mlsanityData to 14 Oct 2026

Missed Connections: Lateral Thinking Puzzles for Large Language Models

arXiv 2404.11730 ↗First posted 17 Apr 2024cs.CL cs.AIcited by 9

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

The Connections puzzle published each day by the New York Times tasks players with dividing a bank of sixteen words into four groups of four words that each relate to a common theme. Solving the puzzle requires both common linguistic knowledge (i.e. definitions and typical usage) as well as, in many cases, lateral or abstract thinking. This is because the four categories ascend in complexity, with the most challenging category often requiring thinking about words in uncommon ways or as parts of larger phrases. We investigate the capacity for automated AI systems to play Connections and explore the game's potential as an automated benchmark for abstract reasoning and a way to measure the semantic information encoded by data-driven linguistic systems. In particular, we study both a sentence-embedding baseline and modern large language models (LLMs). We report their accuracy on the task, measure the impacts of chain-of-thought prompting, and discuss their failure modes. Overall, we find that the Connections task is challenging yet feasible, and a strong test-bed for future work.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 25 of its 33 references that are in the map. First posted on arXiv 17 Apr 2024.

Builds on

25 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  8. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  10. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
  11. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  12. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
11 more
  1. 2022Emergent Analogical Reasoning in Large Language Modelscited by 240
  2. 2019Learning to Speak and Act in a Fantasy Text Adventure Gamecited by 122
  3. 2023Level Generation Through Large Language Modelscited by 68
  4. 2023MarioGPT: Open-Ended Text2Level Generation through Large Language Modelscited by 64
  5. 2016Super Mario as a String: Platformer Level Generation Via LSTMscited by 40
  6. 2020The Chess Transformer: Mastering Play using Generative Language Modelscited by 37
  7. 2023Can Large Language Models Play Text Games Well? Current State-of-the-Art and Open Questionscited by 30
  8. 2023ByteSized32: A Corpus and Challenge Task for Generating Task-Specific World Models Expressed as Text Gamescited by 23
  9. 2021Decrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLPcited by 16
  10. 2020The Go Transformer: Natural Language Modeling for Game Playcited by 12
  11. 2023Solving and Generating NPR Sunday Puzzles with Large Language Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 0; 2026: 1.

9 papers.

  1. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 146
  2. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 30
  3. 2024Weak-eval-Strong: Evaluating and Eliciting Lateral Thinking of LLMs with Situation Puzzlescited by 8
  4. 2024ChatPCG: Large Language Model-Driven Reward Design for Procedural Content Generationcited by 5
  5. 2024NYT-Connections: A Deceptively Simple Text Classification Task that Stumps System-1 Thinkerscited by 4
  6. 2024Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word Gamecited by 3
  7. 2024Non Verbis, Sed Rebus: Large Language Models are Weak Solvers of Italian Rebusescited by 2
  8. 2024LLMs as Method Actors: A Model for Prompt Engineering and Architecturecited by 2
  9. 2026CresOWLve: Benchmarking Creative Problem-Solving Over Real-World Knowledgecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
  2. 2024Augmenting Lateral Thinking in Language Models with Humor and Riddle Data for the BRAINTEASER Task
  3. 2024modeLing: A Novel Dataset for Testing Linguistic Reasoning in Language Models
  4. 2024Puzzle Solving using Reasoning of Large Language Models: A Survey
  5. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models
  6. 2023Large Language Models Are Not Strong Abstract Reasoners
  7. 2023Beyond Words: A Mathematical Framework for Interpreting Large Language Models
  8. 2024Making New Connections: LLMs as Puzzle Generators for The New York Times' Connections Word Game