mlsanityData to 14 Oct 2026

Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualization

arXiv 2206.01532 ↗First posted 3 Jun 2022cs.CL cs.AIcited by 17

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Conceptualization, or viewing entities and situations as instances of abstract concepts in mind and making inferences based on that, is a vital component in human intelligence for commonsense reasoning. Despite recent progress in artificial intelligence to acquire and model commonsense attributed to neural language models and commonsense knowledge graphs (CKGs), conceptualization is yet to be introduced thoroughly, making current approaches ineffective to cover knowledge about countless diverse entities and situations in the real world. To address the problem, we thoroughly study the role of conceptualization in commonsense reasoning, and formulate a framework to replicate human conceptual induction by acquiring abstract knowledge about events regarding abstract concepts, as well as higher-level triples or inferences upon them. We then apply the framework to ATOMIC, a large-scale human-annotated CKG, aided by the taxonomy Probase. We annotate a dataset on the validity of contextualized conceptualizations from ATOMIC on both event and triple levels, develop a series of heuristic rules based on linguistic features, and train a set of neural models to generate and verify abstract knowledge. Based on these components, a pipeline to acquire abstract knowledge is built. A large abstract CKG upon ATOMIC is then induced, ready to be instantiated to infer about unseen entities or situations. Finally, we empirically show the benefits of augmenting CKGs with abstract knowledge in downstream tasks like commonsense inference and zero-shot commonsense QA.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 75 of its 185 references that are in the map. First posted on arXiv 3 Jun 2022.

Builds on

75 of 185 references.

References · 75

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2019Language Models as Knowledge Bases?cited by 1,936
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  11. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  12. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
38 more
  1. 2019How Can We Know What Language Models Know?cited by 959
  2. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  3. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  4. 2019KG-BERT: BERT for Knowledge Graph Completioncited by 324
  5. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
  6. 2019Abductive Commonsense Reasoningcited by 301
  7. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  8. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  9. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 264
  10. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 255
  11. 2020COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphscited by 254
  12. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 249
  13. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  14. 2019KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoningcited by 216
  15. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  16. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  17. 2020A Knowledge-Enhanced Pretraining Model for Commonsense Story Generationcited by 116
  18. 2019ASER: A Large-scale Eventuality Knowledge Graphcited by 89
  19. 2021CEM: Commonsense-aware Empathetic Response Generationcited by 89
  20. 2018Ultra-Fine Entity Typingcited by 87
  21. 2020GLUCOSE: GeneraLized and COntextualized Story Explanationscited by 86
  22. 2018Event2Mind: Commonsense Inference on Events, Intents, and Reactionscited by 82
  23. 2019GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledgecited by 79
  24. 2018A Multi-Axis Annotation Scheme for Event Temporal Relationscited by 75
  25. 2019Do Neural Language Representations Learn Physical Commonsense?cited by 62
  26. 2020TransOMCS: From Linguistic Graphs to Commonsense Knowledgecited by 52
  27. 2020GenericsKB: A Knowledge Base of Generic Statementscited by 51
  28. 2020CSKG: The CommonSense Knowledge Graphcited by 45
  29. 2021Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Modelcited by 39
  30. 2022CICERO: A Dataset for Contextualized Commonsense Inference in Dialoguescited by 38
  31. 2021Modeling Fine-Grained Entity Types with Box Embeddingscited by 36
  32. 2018Constructing Narrative Event Evolutionary Graph for Script Event Predictioncited by 33
  33. 2020Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoningcited by 33
  34. 2021The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Colorcited by 32
  35. 2020Self-supervised Knowledge Triplet Learning for Zero-shot Question Answeringcited by 29
  36. 2021Dimensions of Commonsense Knowledgecited by 29
  37. 2020RICA: Evaluating Robust Inference Capabilities Based on Commonsense Axiomscited by 28
  38. 2021ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualitiescited by 28

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 5; 2025: 1; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023CAR: Conceptualization-Augmented Reasoner for Zero-Shot Commonsense Question Answeringcited by 28
  2. 2023CAT: A Contextualized Conceptualization and Instantiation Framework for Commonsense Reasoningcited by 25
  3. 2022FolkScope: Intention Knowledge Graph Construction for E-commerce Commonsense Discoverycited by 23
  4. 2023AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graphcited by 17
  5. 2024AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimationcited by 14
  6. 2022Automaton-Based Representations of Task Knowledge from Generative Language Modelscited by 12
  7. 2024Abstraction-of-Thought Makes Language Models Better Reasonerscited by 11
  8. 2023Multimodal Pretrained Models for Verifiable Sequential Decision-Making: Planning, Grounding, and Perceptioncited by 8
  9. 2023Harvesting Event Schemas from Large Language Modelscited by 6
  10. 2022PseudoReasoner: Leveraging Pseudo Labels for Commonsense Knowledge Base Populationcited by 5
  11. 2023QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answeringcited by 4
  12. 2023Gold: A Global and Local-aware Denoising Framework for Commonsense Knowledge Graph Noise Detectioncited by 4
5 more
  1. 2024MIKO: Multimodal Intention Knowledge Distillation from Large Language Models for Social-Media Commonsense Discoverycited by 3
  2. 2025ACCESS : A Benchmark for Abstract Causal Event Discovery and Reasoningcited by 3
  3. 2024Zero-shot Commonsense Reasoning over Machine Imaginationcited by 1
  4. 2026Matching Tasks to Objectives: Fine-Tuning and Prompt-Tuning Strategies for Encoder-Decoder Pre-trained Language Modelscited by 1
  5. 2024Modelling Commonsense Commonalities with Multi-Facet Concept Embeddingscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Conceptual and Unbiased Reasoning in Language Models
  2. 2024CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning
  3. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey
  4. 2026Abductive Reasoning with Probabilistic Commonsense
  5. 2023CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models
  6. 2022Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach
  7. 2023Commonsense Knowledge Transfer for Pre-trained Language Models
  8. 2023UNcommonsense Reasoning: Abductive Reasoning about Uncommon Situations