# Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination?

arXiv: https://arxiv.org/abs/2311.09702 · HTML: https://mlsanity.com/paper/2311.09702/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning.
- Builds on 10 of its 28 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Placed in

- LLM reasoning / [Hallucination detection and mitigation](https://mlsanity.com/lr/hallucination-detection-and-mitigation/)

## Builds on (most cited first, 10 of 10)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [Knowledge Crosswords: Geometric Knowledge Reasoning with Large Language Models](https://mlsanity.com/paper/2310.01290/) (2023)

## Built on by (most cited first, 26 of 26)

- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [Disentangling Memory and Reasoning Ability in Large Language Models](https://mlsanity.com/paper/2411.13504/) (2024)
- [Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning](https://mlsanity.com/paper/2502.02871/) (2025)
- [Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning](https://mlsanity.com/paper/2502.09673/) (2025)
- [BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models](https://mlsanity.com/paper/2404.12494/) (2024)
- [NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning](https://mlsanity.com/paper/2505.16022/) (2025)
- [DeepSeek vs. ChatGPT vs. Claude: A Comparative Study for Scientific Computing and Scientific Machine Learning Tasks](https://mlsanity.com/paper/2502.17764/) (2025)
- [AUTOCT: Automating Interpretable Clinical Trial Prediction with LLM Agents](https://mlsanity.com/paper/2506.04293/) (2025)
- [FamiCom: Further Demystifying Prompts for Language Models with Task-Agnostic Performance Estimation](https://mlsanity.com/paper/2406.11243/) (2024)
- [Shortcut Learning in In-Context Learning: A Survey](https://mlsanity.com/paper/2411.02018/) (2024)
- [Vector Policy Optimization: Training for Diversity Improves Test-Time Search](https://mlsanity.com/paper/2605.22817/) (2026)
- [Learning by Analogy: Enhancing Few-Shot Prompting for Math Word Problem Solving with Computational Graph-Based Retrieval](https://mlsanity.com/paper/2411.16454/) (2024)
- [CC-LEARN: Cohort-based Consistency Learning](https://mlsanity.com/paper/2506.15662/) (2025)
- [Familiarity-Aware Evidence Compression for Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.12468/) (2024)
- [Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction](https://mlsanity.com/paper/2410.12040/) (2024)
- [Failure Modes in Multi-Hop QA: The Weakest Link Effect and the Recognition Bottleneck](https://mlsanity.com/paper/2601.12499/) (2026)
- [MARS: A neurosymbolic approach for interpretable drug discovery](https://mlsanity.com/paper/2410.05289/) (2024)
- [Are Transformers Truly Foundational for Robotics?](https://mlsanity.com/paper/2411.16917/) (2024)
- [Measuring and Mitigating Post-hoc Rationalization in Reverse Chain-of-Thought Generation](https://mlsanity.com/paper/2602.14469/) (2026)
- [Are LLMs Really Not Knowledgeable? Mining the Submerged Knowledge in LLMs' Memory](https://mlsanity.com/paper/2412.20846/) (2024)
- [Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment](https://mlsanity.com/paper/2512.09148/) (2025)
- [BARE: Towards Bias-Aware and Reasoning-Enhanced One-Tower Visual Grounding](https://mlsanity.com/paper/2601.01526/) (2026)
- [A Variational Approach for Mitigating Entity Bias in Relation Extraction](https://mlsanity.com/paper/2506.11381/) (2025)
- [Avoiding Knowledge Edit Skipping in Multi-hop Question Answering with Guided Decomposition](https://mlsanity.com/paper/2509.07555/) (2025)
- [Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications](https://mlsanity.com/paper/2510.17764/) (2025)
- [Implicit Reasoning Steering via Concept Chaining](https://mlsanity.com/paper/2607.14242/) (2026)

## Abstract (arXiv)

Despite the recent advancement in large language models (LLMs) and their high performances across numerous benchmarks, recent research has unveiled that LLMs suffer from hallucinations and unfaithful reasoning. This work studies a specific type of hallucination induced by semantic associations. Specifically, we investigate to what extent LLMs take shortcuts from certain keyword/entity biases in the prompt instead of following the correct reasoning path. To quantify this phenomenon, we propose a novel probing method and benchmark called EureQA. We start from questions that LLMs will answer correctly with utmost certainty, and mask the important entity with evidence sentence recursively, asking models to find masked entities according to a chain of evidence before answering the question. During the construction of the evidence, we purposefully replace semantic clues (entities) that may lead to the correct answer with distractor clues (evidence) that will not directly lead to the correct answer but require a chain-like reasoning process. We evaluate if models can follow the correct reasoning chain instead of short-cutting through distractor clues. We find that existing LLMs lack the necessary capabilities to follow correct reasoning paths and resist the attempt of greedy shortcuts. We show that the distractor semantic associations often lead to model hallucination, which is strong evidence that questions the validity of current LLM reasoning.