mlsanityData to 7 Oct 2026

Critical Thinking for Language Models

arXiv 2009.07185 ↗First posted 15 Sep 2020cs.CL cs.AIcited by 31

Not on the map. Why?

Abstract (arXiv)

This paper takes a first step towards a critical thinking curriculum for neural auto-regressive language models. We introduce a synthetic corpus of deductively valid arguments, and generate artificial argumentative texts to train and evaluate GPT-2. Significant transfer learning effects can be observed: Training a model on three simple core schemes allows it to accurately complete conclusions of different, and more complex types of arguments, too. The language models generalize the core argument schemes in a correct way. Moreover, we obtain consistent and promising results for NLU benchmarks. In particular, pre-training on the argument schemes raises zero-shot accuracy on the GLUE diagnostics by up to 15 percentage points. The findings suggest that intermediary pre-training on texts that exemplify basic reasoning abilities (such as typically covered in critical thinking textbooks) might help language models to acquire a broad range of reasoning skills. The synthetic argumentative texts presented in this paper are a promising starting point for building such a "critical thinking curriculum for language models."

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 9; 2023: 10; 2024: 2; 2025: 2; 2026: 2.

31 papers.

Sort:

By area:

  1. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  2. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  3. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  4. 2022FOLIO: Natural Language Reasoning with First-Order Logiccited by 233
  5. 2022Faithful Reasoning Using Large Language Modelscited by 130
  6. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoningcited by 107
  7. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  8. 2022LAMBADA: Backward Chaining for Automated Reasoning in Natural Languagecited by 66
  9. 2023Natural Language Reasoning, A Surveycited by 61
  10. 2023BoardgameQA: A Dataset for Natural Language Reasoning with Contradictory Informationcited by 42
  11. 2021CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Modelscited by 33
  12. 2024Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpuscited by 29
19 more
  1. 2021Flexible Generation of Natural Language Deductionscited by 28
  2. 2022MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoningcited by 22
  3. 2023A Systematic Comparison of Syllogistic Reasoning in Humans and Language Modelscited by 20
  4. 2021Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiabilitycited by 18
  5. 2023Improved Logical Reasoning of Language Models via Differentiable Symbolic Programmingcited by 18
  6. 2023Learning Deductive Reasoning from Synthetic Corpus based on Formal Logiccited by 18
  7. 2021Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2cited by 15
  8. 2022Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safetycited by 13
  9. 2021DeepA2: A Modular Framework for Deep Argument Analysis with Pretrained Neural Text2Text Language Modelscited by 7
  10. 2021Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarkingcited by 7
  11. 2023CLOMO: Counterfactual Logical Modification with Large Language Modelscited by 7
  12. 2023Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogismcited by 6
  13. 2023LogicNet: A Logical Consistency Embedded Face Attribute Learning Networkcited by 4
  14. 2025Are Language Models Efficient Reasoners? A Perspective from Logic Programmingcited by 4
  15. 2022Harnessing Knowledge and Reasoning for Human-Like Natural Language Generation: A Brief Reviewcited by 3
  16. 2025Rationality Check! Benchmarking the Rationality of Large Language Modelscited by 2
  17. 2024ConcEPT: Concept-Enhanced Pre-Training for Language Modelscited by 1
  18. 2026An AI Monkey Gets Grapes for Sure -- Sphere Neural Networks for Reliable Decision-Makingcited by 1
  19. 2026Argument Reconstruction as Supervision for Critical Thinking in LLMscited by 0