mlsanityData to 7 Oct 2026

RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentation

arXiv 2409.16383 ↗First posted 24 Sep 2024cs.CLcited by 3

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Riddle-solving requires advanced reasoning skills, pushing LLMs to engage in abstract thinking and creative problem-solving, often revealing limitations in their cognitive abilities. In this paper, we examine the riddle-solving capabilities of LLMs using a multiple-choice format, exploring how different prompting techniques impact performance on riddles that demand diverse reasoning skills. To enhance results, we introduce RISCORE (RIddle Solving with COntext REcontruciton) a novel fully automated prompting method that generates and utilizes contextually reconstructed sentence-based puzzles in conjunction with the original examples to create few-shot exemplars. Our experiments demonstrate that RISCORE significantly improves the performance of language models in both vertical and lateral thinking tasks, surpassing traditional exemplar selection strategies across a variety of few-shot settings.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 22 of its 41 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

22 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Puzzle Solving using Reasoning of Large Language Models: A Survey17 Feb
  2. 2022FaiRR: Faithful and Robust Deductive Reasoning over Natural Language19 Mar
  3. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?25 Feb

Other references · 19

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  4. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  6. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  7. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  8. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  9. 2022A Survey on In-context Learningcited by 717
  10. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  11. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  12. 2022Reasoning with Language Model Prompting: A Surveycited by 251
7 more
  1. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  2. 2022Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Orderingcited by 158
  3. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  4. 2021RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledgecited by 60
  5. 2022Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processescited by 22
  6. 2021BiRdQA: A Bilingual Dataset for Question Answering on Tricky Riddlescited by 10
  7. 2021A Puzzle-Based Dataset for Natural Language Inferencecited by 4

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Modelscited by 1
  2. 2025The Riddle of Reflection: Evaluating Reasoning and Self-Awareness in Multilingual LLMs using Indian Riddlescited by 1
  3. 2025Can LLMs Solve My Grandma's Riddle? Evaluating Multilingual Large Language Models on Reasoning Traditional Bangla Tricky Riddlescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans
  2. 2025Reasoning Riddles: How Explainability Reveals Cognitive Limits in Vision-Language Models
  3. 2024Augmenting Lateral Thinking in Language Models with Humor and Riddle Data for the BRAINTEASER Task
  4. 2025Probing Large Language Models in Reasoning and Translating Complex Linguistic Puzzles
  5. 2024BloomWise: Enhancing Problem-Solving capabilities of Large Language Models using Bloom's-Taxonomy-Inspired Prompts
  6. 2025RiddleBench: A New Generative Reasoning Benchmark for LLMs
  7. 2024uTeBC-NLP at SemEval-2024 Task 9: Can LLMs be Lateral Thinkers?
  8. 2025Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?