# Teaching Pretrained Models with Commonsense Reasoning: A Preliminary KB-Based Approach

arXiv: https://arxiv.org/abs/1909.09743 · HTML: https://mlsanity.com/paper/1909.09743/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 10 of 10)

- [Explanations from Large Language Models Make Small Reasoners Better](https://mlsanity.com/paper/2210.06726/) (2022)
- [Graph Reasoning for Question Answering with Triplet Retrieval](https://mlsanity.com/paper/2305.18742/) (2023)
- [Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction](https://mlsanity.com/paper/2509.18658/) (2025)
- [Rationalization for Explainable NLP: A Survey](https://mlsanity.com/paper/2301.08912/) (2023)
- [RNR: Teaching Large Language Models to Follow Roles and Rules](https://mlsanity.com/paper/2409.13733/) (2024)
- [Can Pretrained Language Models (Yet) Reason Deductively?](https://mlsanity.com/paper/2210.06442/) (2022)
- [Commonsense Knowledge Transfer for Pre-trained Language Models](https://mlsanity.com/paper/2306.02388/) (2023)
- [Unifying Structure Reasoning and Language Model Pre-training for Complex Reasoning](https://mlsanity.com/paper/2301.08913/) (2023)
- [Discourse-Aware Graph Networks for Textual Logical Reasoning](https://mlsanity.com/paper/2207.01450/) (2022)
- [Prompt and Parameter Co-Optimization for Large Language Models](https://mlsanity.com/paper/2509.24245/) (2025)

## Abstract (arXiv)

Recently, pretrained language models (e.g., BERT) have achieved great success on many downstream natural language understanding tasks and exhibit a certain level of commonsense reasoning ability. However, their performance on commonsense tasks is still far from that of humans. As a preliminary attempt, we propose a simple yet effective method to teach pretrained models with commonsense reasoning by leveraging the structured knowledge in ConceptNet, the largest commonsense knowledge base (KB). Specifically, the structured knowledge in KB allows us to construct various logical forms, and then generate multiple-choice questions requiring commonsense logical reasoning. Experimental results demonstrate that, when refined on these training examples, the pretrained models consistently improve their performance on tasks that require commonsense reasoning, especially in the few-shot learning setting. Besides, we also perform analysis to understand which logical relations are more relevant to commonsense reasoning.