mlsanityData to 14 Oct 2026

Incorporating Commonsense Knowledge Graph in Pretrained Models for Social Commonsense Tasks

arXiv 2105.05457 ↗First posted 12 May 2021cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Pretrained language models have excelled at many NLP tasks recently; however, their social intelligence is still unsatisfactory. To enable this, machines need to have a more general understanding of our complicated world and develop the ability to perform commonsense reasoning besides fitting the specific downstream tasks. External commonsense knowledge graphs (KGs), such as ConceptNet, provide rich information about words and their relationships. Thus, towards general commonsense learning, we propose two approaches to implicitly and explicitly infuse such KGs into pretrained language models. We demonstrate our proposed methods perform well on SocialIQA, a social commonsense reasoning task, in both limited and full training data regimes.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 12 May 2021.

Builds on

17 of 29 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  6. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  7. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  8. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  9. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  10. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 285
  11. 2016Bidirectional Attention Flow for Machine Comprehensioncited by 278
  12. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
5 more
  1. 2020Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graphcited by 57
  2. 2019How Additional Knowledge can Improve Natural Language Commonsense Question Answering?cited by 22
  3. 2019Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answeringcited by 16
  4. 2018Yuanfudao at SemEval-2018 Task 11: Three-way Attention and Relational Knowledge for Commonsense Machine Comprehensioncited by 12
  5. 2020Generating Commonsense Explanation by Extracting Bridge Concepts from Reasoning Pathscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 9; 2024: 3; 2025: 0; 2026: 1.

24 papers.

Sort:

By area:

  1. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  2. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 214
  3. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  4. 2023NormBank: A Knowledge Bank of Situational Social Normscited by 60
  5. 2023Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modelingcited by 56
  6. 2022Rainier: Reinforced Knowledge Introspector for Commonsense Question Answeringcited by 43
  7. 2021Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attentioncited by 40
  8. 2022Kformer: Knowledge Injection in Transformer Feed-Forward Layerscited by 37
  9. 2023PeaCoK: Persona Commonsense Knowledge for Consistent and Engaging Narrativescited by 19
  10. 2022Mind the Gap! Injecting Commonsense Knowledge for Abstractive Dialogue Summarizationcited by 14
  11. 2024Tox-BART: Leveraging Toxicity Attributes for Explanation Generation of Implicit Hate Speechcited by 11
  12. 2022Elaboration-Generating Commonsense Question Answering at Scalecited by 9
12 more
  1. 2023Text-Augmented Open Knowledge Graph Completion via Pre-Trained Language Modelscited by 8
  2. 2024Knowledge Graph Tuning: Real-time Large Language Model Personalization based on Human Feedbackcited by 6
  3. 2022Modularized Transfer Learning with Multiple Knowledge Graphs for Zero-shot Commonsense Reasoningcited by 5
  4. 2022A Survey on Knowledge-Enhanced Pre-trained Language Modelscited by 3
  5. 2023Text-To-KG Alignment: Comparing Current Methods on Classification Taskscited by 2
  6. 2021Semantic Categorization of Social Knowledge for Commonsense Question Answeringcited by 1
  7. 2023IEKG: A Commonsense Knowledge Graph for Idiomatic Expressionscited by 1
  8. 2024Rule or Story, Which is a Better Commonsense Expression for Talking with Large Language Models?cited by 1
  9. 2021Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systemscited by 0
  10. 2023Open-ended Commonsense Reasoning with Unrestricted Answer Scopecited by 0
  11. 2023Moral Sparks in Social Media Narrativescited by 0
  12. 2026Text2Model: Modeling Copilots for Text-to-Model Translationcited by 0