mlsanityData to 7 Oct 2026

A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners

arXiv 2406.11050 ↗First posted 16 Jun 2024cs.CL cs.AIcited by 73

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This study introduces a hypothesis-testing framework to assess whether large language models (LLMs) possess genuine reasoning abilities or primarily depend on token bias. We go beyond evaluating LLMs on accuracy; rather, we aim to investigate their token bias in solving logical reasoning tasks. Specifically, we develop carefully controlled synthetic datasets, featuring conjunction fallacy and syllogistic problems. Our framework outlines a list of hypotheses where token biases are readily identifiable, with all null hypotheses assuming genuine reasoning capabilities of LLMs. The findings in this study suggest, with statistical guarantee, that most LLMs still struggle with logical reasoning. While they may perform well on classic problems, their success largely depends on recognizing superficial patterns with strong token bias, thereby raising concerns about their actual reasoning and generalization abilities. Codes and data are open-sourced at https://github.com/bowen-upenn/llm_token_bias.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 42 of its 74 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

42 of 74 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Puzzle Solving using Reasoning of Large Language Models: A Survey17 Feb
  2. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning14 Nov
  3. 2023At Which Training Stage Does Code Data Help LLMs Reasoning?28 Sep
  4. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance26 May
  5. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
  6. 2022Thinking Fast and Slow in Large Language Models10 Dec
  7. 2022Language models show human-like content effects on reasoning tasks14 Jul
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?25 Feb

Other references · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  9. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
  10. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  11. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  12. 2023Reasoning with Language Model is Planning with World Modelcited by 846
22 more
  1. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  2. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  3. 2022Using cognitive psychology to understand GPT-3cited by 389
  4. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 288
  5. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  6. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 221
  7. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  8. 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
  9. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 172
  10. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  11. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
  12. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  13. 2022Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrationscited by 70
  14. 2022A Logic for Expressing Log-Precision Transformerscited by 67
  15. 2020Temporal Reasoning on Implicit Events from Distant Supervisioncited by 58
  16. 2023Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5cited by 41
  17. 2023Challenging the appearance of machine intelligence: Cognitive bias in LLMs and Best Practices for Adoptioncited by 21
  18. 2022Learning to Decompose: Hypothetical Question Decomposition Based on Comparable Textscited by 18
  19. 2023Unveiling Theory of Mind in Large Language Models: A Parallel to Single Neurons in the Human Braincited by 18
  20. 2024WildfireGPT: Tailored Large Language Model for Wildfire Analysiscited by 15
  21. 2024Will the Real Linda Please Stand up...to Large Language Models? Examining the Representativeness Heuristic in LLMscited by 12
  22. 2024Heuristic Reasoning in AI: Instrumental Use and Mimetic Absorptioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 46; 2026: 15.

73 papers.

Sort:

By area:

  1. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  4. 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 120
  5. 2024On Memorization of Large Language Models in Logical Reasoningcited by 114
  6. 2025Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinkingcited by 56
  7. 2025SAKURA: On the Multi-hop Reasoning of Large Audio-Language Models Based on Speech and Audio Informationcited by 42
  8. 2024Evaluating the Robustness of Analogical Reasoning in Large Language Modelscited by 31
  9. 2024Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpuscited by 29
  10. 2025Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignmentcited by 22
  11. 2025Frontier LLMs Still Struggle with Simple Reasoning Taskscited by 20
  12. 2026Large Language Model Reasoning Failurescited by 20
38 more
  1. 2024Evaluating Modern Approaches in 3D Scene Reconstruction: NeRF vs Gaussian-Based Methodscited by 16
  2. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinkingcited by 14
  3. 2024GIVE: Structured Reasoning of Large Language Models with Knowledge Graph Inspired Veracity Extrapolationcited by 13
  4. 2025MARS: toward more efficient multi-agent collaboration for LLM reasoningcited by 13
  5. 2024MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofscited by 12
  6. 2024CBEval: A framework for evaluating and interpreting cognitive biases in LLMscited by 12
  7. 2025TreeCut: A Synthetic Unanswerable Math Word Problem Dataset for LLM Hallucination Evaluationcited by 11
  8. 2025Lost in Cultural Translation: Do LLMs Struggle with Math Across Cultural Contexts?cited by 9
  9. 2024In-Context Learning May Not Elicit Trustworthy Reasoning: A-Not-B Errors in Pretrained Language Modelscited by 8
  10. 2026One Model, All Roles: Multi-Turn, Multi-Agent Self-Play Reinforcement Learning for Conversational Social Intelligencecited by 7
  11. 2025Beyond Statistical Learning: Exact Learning Is Essential for General Intelligencecited by 6
  12. 2025Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generationcited by 6
  13. 2024Challenges and Future Directions of Data-Centric AI Alignmentcited by 5
  14. 2025None of the Others: a General Technique to Distinguish Reasoning from Memorization in Multiple-Choice LLM Evaluation Benchmarkscited by 5
  15. 2025ASyMOB: Algebraic Symbolic Mathematical Operations Benchmarkcited by 5
  16. 2025Do Large Language Models Reason Causally Like Us? Even Better?cited by 4
  17. 2025Benchmarking Abstract and Reasoning Abilities Through A Theoretical Perspectivecited by 4
  18. 2025Garbage In, Reasoning Out? Why Benchmark Scores are Unreliable and What to Do About Itcited by 4
  19. 2025Event-Driven Storytelling with Multiple Lifelike Humans in a 3D Scenecited by 4
  20. 2026Investigating Social Bias Changes in Quantized Language Modelscited by 4
  21. 2026Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluationcited by 4
  22. 2024Beyond Surface Structure: A Causal Assessment of LLMs' Comprehension Abilitycited by 3
  23. 2025Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More Morecited by 3
  24. 2025Exploring the Hidden Reasoning Process of Large Language Models by Misleading Themcited by 3
  25. 2025Socratic Chart: Cooperating Multiple Agents for Robust SVG Chart Understandingcited by 3
  26. 2025Deliberate Planning in Language Models with Symbolic Representationcited by 3
  27. 2025RocqStar: Leveraging Similarity-driven Retrieval and Agentic Systems for Rocq generationcited by 3
  28. 2025TRIM: Token-wise Attention-Derived Saliency for Data-Efficient Instruction Tuningcited by 3
  29. 2025Numerical Sensitivity and Robustness: Exploring the Flaws of Mathematical Reasoning in Large Language Modelscited by 3
  30. 2025On the Reasoning Capacity of AI Models and How to Quantify Itcited by 2
  31. 2025On the Notion that Language Models Reasoncited by 2
  32. 2026EMS: Multi-Agent Voting via Efficient Majority-then-Stoppingcited by 2
  33. 2026HCP-MAD:Heterogeneous Consensus-Progressive Reasoning for Efficient Multi-Agent Debatecited by 2
  34. 2025Does a Large Language Model Really Speak in Human-Like Language?cited by 1
  35. 2025Three tiers of computation in transformers and in brain architecturescited by 1
  36. 2025Enigme: Generative Text Puzzles for Evaluating Reasoning in Language Modelscited by 1
  37. 2025CAMA: Enhancing Mathematical Reasoning in Large Language Models with Causal Knowledgecited by 1
  38. 2025Robust Native Language Identification through Agentic Decompositioncited by 1

50 most cited of 73 All 73 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Evaluting Fake Reasoning Bias in Language Models
  2. 2023Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond
  3. 2025Investigating the Robustness of Deductive Reasoning with Large Language Models
  4. 2025Reasoning Capabilities and Invariability of Large Language Models
  5. 2025Position: On the Methodological Pitfalls of Evaluating Base LLMs for Reasoning
  6. 2024Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset
  7. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
  8. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey