# CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning

arXiv: https://arxiv.org/abs/2401.07286 · HTML: https://mlsanity.com/paper/2401.07286/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 10 of its 90 references that are in the map.
- First posted on arXiv 14 Jan 2024.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Larger language models do in-context learning differently](https://mlsanity.com/paper/2303.03846/) (2023)
- [AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph](https://mlsanity.com/paper/2311.09174/) (2023)
- [On the Role of Conceptualization in Commonsense Knowledge Graph Construction](https://mlsanity.com/paper/2003.03239/) (2020)
- [Multi-hop Commonsense Knowledge Injection Framework for Zero-Shot Commonsense Question Answering](https://mlsanity.com/paper/2305.05936/) (2023)

## Built on by (most cited first, 28 of 28)

- [NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding](https://mlsanity.com/paper/2404.13627/) (2024)
- [GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory](https://mlsanity.com/paper/2406.11149/) (2024)
- [AutoSchemaKG: Autonomous Knowledge Graph Construction through Dynamic Schema Induction from Web-Scale Corpora](https://mlsanity.com/paper/2505.23628/) (2025)
- [Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information](https://mlsanity.com/paper/2408.02559/) (2024)
- [ECon: On the Detection and Resolution of Evidence Conflicts](https://mlsanity.com/paper/2410.04068/) (2024)
- [Revisiting Epistemic Markers in Confidence Estimation: Can Markers Accurately Reflect Large Language Models' Uncertainty?](https://mlsanity.com/paper/2505.24778/) (2025)
- [MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset](https://mlsanity.com/paper/2406.02106/) (2024)
- [IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce](https://mlsanity.com/paper/2406.10173/) (2024)
- [MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding](https://mlsanity.com/paper/2406.10701/) (2024)
- [EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association](https://mlsanity.com/paper/2505.15196/) (2025)
- [RedDebate: Safer Responses Through Multi-Agent Red Teaming Debates](https://mlsanity.com/paper/2506.11083/) (2025)
- [MuSLR: Multimodal Symbolic Logical Reasoning](https://mlsanity.com/paper/2509.25851/) (2025)
- [Intention Knowledge Graph Construction for User Intention Relation Modeling](https://mlsanity.com/paper/2412.11500/) (2024)
- [arXiv2Table: Toward Realistic Benchmarking and Evaluation for LLM-Based Literature-Review Table Generation](https://mlsanity.com/paper/2504.10284/) (2025)
- [SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding](https://mlsanity.com/paper/2507.20185/) (2025)
- [Cooper: Co-Optimizing Policy and Reward Models in Reinforcement Learning for Large Language Models](https://mlsanity.com/paper/2508.05613/) (2025)
- [COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes](https://mlsanity.com/paper/2409.04053/) (2024)
- [Concept-Reversed Winograd Schema Challenge: Evaluating and Improving Robust Reasoning in Large Language Models via Abstraction](https://mlsanity.com/paper/2410.12040/) (2024)
- [HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning](https://mlsanity.com/paper/2601.22448/) (2026)
- [MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discovery](https://mlsanity.com/paper/2402.18169/) (2024)
- [On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions](https://mlsanity.com/paper/2406.10885/) (2024)
- [Robust Misinformation Detection by Visiting Potential Commonsense Conflict](https://mlsanity.com/paper/2504.21604/) (2025)
- [EcomEdit: An Automated E-commerce Knowledge Editing Framework for Enhanced Product and Purchase Intention Understanding](https://mlsanity.com/paper/2410.14276/) (2024)
- [ConKE: Conceptualization-Augmented Knowledge Editing in Large Language Models for Commonsense Reasoning](https://mlsanity.com/paper/2412.11418/) (2024)
- [XToM: Exploring the Multilingual Theory of Mind for Large Language Models](https://mlsanity.com/paper/2506.02461/) (2025)
- [Structuring the Unstructured: A Systematic Review of Text-to-Structure Generation for Agentic AI with a Universal Evaluation Framework](https://mlsanity.com/paper/2508.12257/) (2025)
- [Semiotic logical hexagon theory for LLM logical reasoning](https://mlsanity.com/paper/2607.21933/) (2026)
- [Persona Knowledge-Aligned Prompt Tuning Method for Online Debate](https://mlsanity.com/paper/2410.04239/) (2024)

## Abstract (arXiv)

The sequential process of conceptualization and instantiation is essential to generalizable commonsense reasoning as it allows the application of existing knowledge to unfamiliar scenarios. However, existing works tend to undervalue the step of instantiation and heavily rely on pre-built concept taxonomies and human annotations to collect both types of knowledge, resulting in a lack of instantiated knowledge to complete reasoning, high cost, and limited scalability. To tackle these challenges, we introduce CANDLE, a distillation framework that iteratively performs contextualized conceptualization and instantiation over commonsense knowledge bases by instructing large language models to generate both types of knowledge with critic filtering. By applying CANDLE to ATOMIC, we construct a comprehensive knowledge base comprising six million conceptualizations and instantiated commonsense knowledge triples. Both types of knowledge are firmly rooted in the original ATOMIC dataset, and intrinsic evaluations demonstrate their exceptional quality and diversity. Empirical results indicate that distilling CANDLE on student models provides benefits across four downstream tasks. Our code, data, and models are publicly available at https://github.com/HKUST-KnowComp/CANDLE.