# ItD: Large Language Models Can Teach Themselves Induction through Deduction

arXiv: https://arxiv.org/abs/2403.05789 · HTML: https://mlsanity.com/paper/2403.05789/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 9 Mar 2024.

## Builds on (most cited first, 16 of 16)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Large Language Models as General Pattern Machines](https://mlsanity.com/paper/2307.04721/) (2023)
- [Instruction Induction: From Few Examples to Natural Language Task Descriptions](https://mlsanity.com/paper/2205.10782/) (2022)
- [Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey](https://mlsanity.com/paper/2305.18703/) (2023)
- [Hypothesis Search: Inductive Reasoning with Language Models](https://mlsanity.com/paper/2309.05660/) (2023)
- [Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement](https://mlsanity.com/paper/2310.08559/) (2023)
- [Large Language Models can Learn Rules](https://mlsanity.com/paper/2310.07064/) (2023)
- [Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners](https://mlsanity.com/paper/2305.14825/) (2023)
- [Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks](https://mlsanity.com/paper/2311.09247/) (2023)
- [HYTREL: Hypergraph-enhanced Tabular Data Representation Learning](https://mlsanity.com/paper/2307.08623/) (2023)
- [Failures Pave the Way: Enhancing Large Language Models through Tuning-free Rule Accumulation](https://mlsanity.com/paper/2310.15746/) (2023)
- [Guideline Learning for In-context Information Extraction](https://mlsanity.com/paper/2310.05066/) (2023)

## Built on by (most cited first, 8 of 8)

- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning](https://mlsanity.com/paper/2410.04691/) (2024)
- [LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning](https://mlsanity.com/paper/2505.07437/) (2025)
- [A Survey of Inductive Reasoning for Large Language Models](https://mlsanity.com/paper/2510.10182/) (2025)
- [Inductive-Deductive Strategy Reuse for Multi-Turn Instructional Dialogues](https://mlsanity.com/paper/2404.11095/) (2024)
- [RAISE: Reinforced Adaptive Instruction Selection For Large Language Models](https://mlsanity.com/paper/2504.07282/) (2025)
- [Shuttle Between the Instructions and the Parameters of Large Language Models](https://mlsanity.com/paper/2502.02315/) (2025)
- [Don't Let Me Ask for It: LLMs Show Deficiencies in Active Multi-Turn Information Acquisition for Abductive Inference](https://mlsanity.com/paper/2608.03388/) (2026)

## Abstract (arXiv)

Although Large Language Models (LLMs) are showing impressive performance on a wide range of Natural Language Processing tasks, researchers have found that they still have limited ability to conduct induction. Recent works mainly adopt ``post processes'' paradigms to improve the performance of LLMs on induction (e.g., the hypothesis search & refinement methods), but their performance is still constrained by the inherent inductive capability of the LLMs. In this paper, we propose a novel framework, Induction through Deduction (ItD), to enable the LLMs to teach themselves induction through deduction. The ItD framework is composed of two main components: a Deductive Data Generation module to generate induction data and a Naive Bayesian Induction module to optimize the fine-tuning and decoding of LLMs. Our empirical results showcase the effectiveness of ItD on two induction benchmarks, achieving relative performance improvement of 36% and 10% compared with previous state-of-the-art, respectively. Our ablation study verifies the effectiveness of two key modules of ItD. We also verify the effectiveness of ItD across different LLMs and deductors. The data and code of this paper can be found at https://anonymous.4open.science/r/ItD-E844.