mlsanityData to 7 Oct 2026

Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey

arXiv 2201.12438 ↗First posted 28 Jan 2022cs.CLcited by 38

Not on the map. Why?

Abstract (arXiv)

While commonsense knowledge acquisition and reasoning has traditionally been a core research topic in the knowledge representation and reasoning community, recent years have seen a surge of interest in the natural language processing community in developing pre-trained models and testing their ability to address a variety of newly designed commonsense knowledge reasoning and generation tasks. This paper presents a survey of these tasks, discusses the strengths and weaknesses of state-of-the-art pre-trained models for commonsense reasoning and generation as revealed by these tasks, and reflects on future research directions.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 56 of its 84 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

56 of 84 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  8. 2019Language Models as Knowledge Bases?cited by 1,922
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  10. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  11. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  12. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
38 more
  1. 2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
  2. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  3. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  4. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  5. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  6. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  7. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  8. 2019Abductive Commonsense Reasoningcited by 300
  9. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  10. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  11. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  12. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  13. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 194
  14. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  15. 2020Adversarial Filters of Dataset Biasescited by 159
  16. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  17. 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
  18. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  19. 2019"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understandingcited by 121
  20. 2020Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Modelscited by 110
  21. 2019Evaluating Commonsense in Pre-trained Language Modelscited by 95
  22. 2017Generating Sentences by Editing Prototypescited by 88
  23. 2019Barack's Wife Hillary: Using Knowledge-Graphs for Fact-Aware Language Modelingcited by 71
  24. 2020Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoningcited by 65
  25. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  26. 2019Do Neural Language Representations Learn Physical Commonsense?cited by 62
  27. 2019Commonsense Knowledge Base Completion with Structural and Semantic Contextcited by 61
  28. 2017Thinking Fast and Slow: Optimization Decomposition Across Timescalescited by 60
  29. 2021RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledgecited by 60
  30. 2020Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graphcited by 57
  31. 2019Non-Monotonic Sequential Text Generationcited by 55
  32. 2021TIMEDIAL: Temporal Commonsense Reasoning in Dialogcited by 55
  33. 2019Pythia: AI-assisted Code Completion Systemcited by 54
  34. 2021PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D Worldcited by 47
  35. 2020Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphscited by 46
  36. 2020Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answeringcited by 44
  37. 2020Temporal Common Sense Acquisition with Minimal Supervisioncited by 38
  38. 2019What Does My QA Model Know? Devising Controlled Probes using Expert Knowledgecited by 19

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 20; 2024: 6; 2025: 5; 2026: 1.

38 papers.

Sort:

By area:

  1. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  2. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  3. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  4. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  5. 2023ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Modelscited by 63
  6. 2023Large Language Models in Education: Vision and Opportunitiescited by 54
  7. 2023Benchmarks for Automated Commonsense Reasoning: A Surveycited by 40
  8. 2023Editing Common Sense in Transformerscited by 28
  9. 2023Through the Lens of Core Competency: Survey on Evaluation of Large Language Modelscited by 12
  10. 2023Evaluating the Deductive Competence of Large Language Modelscited by 12
  11. 2023An Overview Of Temporal Commonsense Reasoning and Acquisitioncited by 10
  12. 2022Negation, Coordination, and Quantifiers in Contextualized Language Modelscited by 9
26 more
  1. 2023Knowledge-augmented Few-shot Visual Relation Detectioncited by 5
  2. 2024The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoningcited by 5
  3. 2024Belief in the Machine: Investigating Epistemological Blind Spots of Language Modelscited by 5
  4. 2023The curse of language biases in remote sensing VQA: the role of spatial attributes, language diversity, and the need for clear evaluationcited by 4
  5. 2024CASA: Causality-driven Argument Sufficiency Assessmentcited by 4
  6. 2023BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answeringcited by 3
  7. 2023SituatedGen: Incorporating Geographical and Temporal Contexts into Generative Commonsense Reasoningcited by 3
  8. 2023Language Models as Knowledge Bases for Visual Word Sense Disambiguationcited by 3
  9. 2023CORECODE: A Common Sense Annotated Dialogue Dataset with Benchmark Tasks for Chinese Large Language Modelscited by 3
  10. 2024PARADISE: Evaluating Implicit Planning Skills of Language Models with Procedural Warnings and Tips Datasetcited by 3
  11. 2025Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Frameworkcited by 3
  12. 2025AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarioscited by 3
  13. 2023The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challengescited by 2
  14. 2023A Formalism and Approach for Improving Robustness of Large Language Models Using Risk-Adjusted Confidence Scorescited by 2
  15. 2024Temporal Validity Change Predictioncited by 2
  16. 2023Large Language Models Are Also Good Prototypical Commonsense Reasonerscited by 1
  17. 2023Ontology Revision based on Pre-trained Language Modelscited by 1
  18. 2023IEKG: A Commonsense Knowledge Graph for Idiomatic Expressionscited by 1
  19. 2024Defining and Evaluating Decision and Composite Risk in Language Models Applied to Natural Language Inferencecited by 1
  20. 2025Randomly Sampled Language Reasoning Problems Elucidate Limitations of In-Context Learningcited by 1
  21. 2025Do LLMs exhibit the same commonsense capabilities across languages?cited by 1
  22. 2026Exploring the Meta-level Reasoning of Large Language Models via a Tool-based Multi-hop Tabular Question Answering Taskcited by 1
  23. 2022Changes from Classical Statistics to Modern Statistics and Data Sciencecited by 0
  24. 2022A Probabilistic-Logic based Commonsense Representation Framework for Modelling Inferences with Multiple Antecedents and Varying Likelihoodscited by 0
  25. 2022TegFormer: Topic-to-Essay Generation with Good Topic Coverage and High Text Coherencecited by 0
  26. 2025CCQA: Generating Question from Solution Can Improve Inference-Time Reasoning in SLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach
  2. 2023Advancing Transformer's Capabilities in Commonsense Reasoning
  3. 2023Commonsense Knowledge Transfer for Pre-trained Language Models
  4. 2023Knowledge Graph-Augmented Korean Generative Commonsense Reasoning
  5. 2022Probing Commonsense Knowledge in Pre-trained Language Models with Sense-level Precision and Expanded Vocabulary
  6. 2022CIKQA: Learning Commonsense Inference with a Unified Knowledge-in-the-loop QA Paradigm
  7. 2022Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts
  8. 2022Language Models of Code are Few-Shot Commonsense Learners