# Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations

arXiv: https://arxiv.org/abs/2502.16169 · HTML: https://mlsanity.com/paper/2502.16169/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 22 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 12 of 12)

- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models](https://mlsanity.com/paper/2410.05229/) (2024)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [On the Measure of Intelligence](https://mlsanity.com/paper/1911.01547/) (2019)
- [MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbations](https://mlsanity.com/paper/2502.06453/) (2025)
- [BUSTLE: Bottom-Up Program Synthesis Through Learning-Guided Exploration](https://mlsanity.com/paper/2007.14381/) (2020)
- [Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs](https://mlsanity.com/paper/2408.00114/) (2024)
- [ChatLog: Carefully Evaluating the Evolution of ChatGPT Across Time](https://mlsanity.com/paper/2304.14106/) (2023)
- [LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning](https://mlsanity.com/paper/2502.11176/) (2025)

## Built on by (most cited first, 12 of 12)

- [The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning](https://mlsanity.com/paper/2504.05081/) (2025)
- [WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Quality](https://mlsanity.com/paper/2510.18560/) (2025)
- [Climplicit: Climatic Implicit Embeddings for Global Ecological Tasks](https://mlsanity.com/paper/2504.05089/) (2025)
- [LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Game](https://mlsanity.com/paper/2510.04980/) (2025)
- [Contextual Drag: How Errors in the Context Affect LLM Reasoning](https://mlsanity.com/paper/2602.04288/) (2026)
- [Large Language Models Develop Novel Social Biases Through Adaptive Exploration](https://mlsanity.com/paper/2511.06148/) (2025)
- [Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs](https://mlsanity.com/paper/2506.17630/) (2025)
- [Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models](https://mlsanity.com/paper/2604.18612/) (2026)
- [Knowledge Vector of Logical Reasoning in Large Language Models](https://mlsanity.com/paper/2604.23877/) (2026)
- [Lifted State Hypothesis in Large Language Models](https://mlsanity.com/paper/2607.19360/) (2026)
- [CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs](https://mlsanity.com/paper/2608.14791/) (2026)
- [SIMLIFE: Pattern Understanding for Long-Horizon Human-Agent Partnership](https://mlsanity.com/paper/2609.19610/) (2026)

## Abstract (arXiv)

Inductive reasoning, a cornerstone of human cognition, enables generalization from limited data but hasn't yet been fully achieved by large language models (LLMs). While modern LLMs excel at reasoning tasks, their ability to maintain stable and consistent rule abstraction under imperfect observations remains underexplored. To fill this gap, in this work, we introduce Robust Rule Induction, a task that evaluates LLMs' capability in inferring rules from data that are fused with noisy examples. To address this task, we further propose Sample-steered Rule Refinement (SRR), a method enhancing reasoning stability via observation diversification and execution-guided feedback. Experiments across arithmetic, cryptography, and list functions reveal: (1) SRR outperforms other methods with minimal performance degradation under noise; (2) Despite slight accuracy variation, LLMs exhibit instability under noise (e.g., 0% accuracy change with only 70% consistent score); (3) Counterfactual task gaps highlight LLMs' reliance on memorized patterns over genuine abstraction. Our findings challenge LLMs' reasoning robustness, revealing susceptibility to hypothesis drift and pattern overfitting, while providing empirical evidence critical for developing human-like inductive systems. Code and data are available at https://github.com/HKUST-KnowComp/Robust-Rule-Induction.