# Critical Thinking for Language Models

arXiv: https://arxiv.org/abs/2009.07185 · HTML: https://mlsanity.com/paper/2009.07185/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2020.

## Built on by (most cited first, 30 of 31)

- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought](https://mlsanity.com/paper/2210.01240/) (2022)
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022)
- [FOLIO: Natural Language Reasoning with First-Order Logic](https://mlsanity.com/paper/2209.00840/) (2022)
- [Faithful Reasoning Using Large Language Models](https://mlsanity.com/paper/2208.14271/) (2022)
- [GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning](https://mlsanity.com/paper/2312.12241/) (2023)
- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [LAMBADA: Backward Chaining for Automated Reasoning in Natural Language](https://mlsanity.com/paper/2212.13894/) (2022)
- [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023)
- [BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Information](https://mlsanity.com/paper/2306.07934/) (2023)
- [CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models](https://mlsanity.com/paper/2112.11941/) (2021)
- [Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus](https://mlsanity.com/paper/2411.12498/) (2024)
- [Flexible Generation of Natural Language Deductions](https://mlsanity.com/paper/2104.08825/) (2021)
- [MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning](https://mlsanity.com/paper/2203.00357/) (2022)
- [A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models](https://mlsanity.com/paper/2311.00445/) (2023)
- [Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiability](https://mlsanity.com/paper/2112.09054/) (2021)
- [Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming](https://mlsanity.com/paper/2305.03742/) (2023)
- [Learning Deductive Reasoning from Synthetic Corpus based on Formal Logic](https://mlsanity.com/paper/2308.07336/) (2023)
- [Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2](https://mlsanity.com/paper/2103.13033/) (2021)
- [Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safety](https://mlsanity.com/paper/2212.06295/) (2022)
- [DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Models](https://mlsanity.com/paper/2110.01509/) (2021)
- [Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarking](https://mlsanity.com/paper/2112.00086/) (2021)
- [CLOMO: Counterfactual Logical Modification with Large Language Models](https://mlsanity.com/paper/2311.17438/) (2023)
- [Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism](https://mlsanity.com/paper/2310.14868/) (2023)
- [LogicNet: A Logical Consistency Embedded Face Attribute Learning Network](https://mlsanity.com/paper/2311.11208/) (2023)
- [Are Language Models Efficient Reasoners? A Perspective from Logic Programming](https://mlsanity.com/paper/2510.25626/) (2025)
- [Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Review](https://mlsanity.com/paper/2212.03747/) (2022)
- [Rationality Check! Benchmarking the Rationality of Large Language Models](https://mlsanity.com/paper/2509.14546/) (2025)
- [ConcEPT: Concept-Enhanced Pre-Training for Language Models](https://mlsanity.com/paper/2401.05669/) (2024)
- [An AI Monkey Gets Grapes for Sure -- Sphere Neural Networks for Reliable Decision-Making](https://mlsanity.com/paper/2601.00142/) (2026)

## Abstract (arXiv)

This paper takes a first step towards a critical thinking curriculum for neural auto-regressive language models. We introduce a synthetic corpus of deductively valid arguments, and generate artificial argumentative texts to train and evaluate GPT-2. Significant transfer learning effects can be observed: Training a model on three simple core schemes allows it to accurately complete conclusions of different, and more complex types of arguments, too. The language models generalize the core argument schemes in a correct way. Moreover, we obtain consistent and promising results for NLU benchmarks. In particular, pre-training on the argument schemes raises zero-shot accuracy on the GLUE diagnostics by up to 15 percentage points. The findings suggest that intermediary pre-training on texts that exemplify basic reasoning abilities (such as typically covered in critical thinking textbooks) might help language models to acquire a broad range of reasoning skills. The synthetic argumentative texts presented in this paper are a promising starting point for building such a "critical thinking curriculum for language models."