mlsanityData to 7 Oct 2026

Commonsense Knowledge Mining from Pretrained Models

arXiv 1909.00505 ↗First posted 2 Sep 2019cs.CL cs.AI cs.LGcited by 189

Not on the map. Why?

Abstract (arXiv)

Inferring commonsense knowledge is a key challenge in natural language processing, but due to the sparsity of training data, previous work has shown that supervised methods for commonsense knowledge mining underperform when evaluated on novel data. In this work, we develop a method for generating commonsense knowledge using a large, pre-trained bidirectional language model. By transforming relational triples into masked sentences, we can use this model to rank a triple's validity by the estimated pointwise mutual information between the two entities. Since we do not update the weights of the bidirectional model, our approach is not biased by the coverage of any one commonsense knowledge base. Though this method performs worse on a test set than models explicitly trained on a corresponding training set, it outperforms these methods when mining commonsense knowledge from new sources, suggesting that unsupervised techniques may generalize better than current supervised approaches.

Key facts. Cited by 189 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 55; 2022: 52; 2023: 45; 2024: 21; 2025: 9; 2026: 7.

189 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  3. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  5. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  6. 2021GPT Understands, Toocited by 715
  7. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  8. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  9. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  10. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  11. 2021Symbolic Knowledge Distillation: from General Language Models to Commonsense Modelscited by 270
  12. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
38 more
  1. 2022Reasoning with Language Model Prompting: A Surveycited by 251
  2. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  3. 2021PPT: Pre-trained Prompt Tuning for Few-shot Learningcited by 232
  4. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  5. 2021Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classificationcited by 183
  6. 2021KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extractioncited by 180
  7. 2022A Review on Language Models as Knowledge Basescited by 156
  8. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  9. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  10. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  11. 2021Entailment as Few-Shot Learnercited by 122
  12. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  13. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  14. 2023Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Modelscited by 117
  15. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  16. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  17. 2023Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Modelscited by 80
  18. 2023ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planningcited by 77
  19. 2022Black-box Prompt Learning for Pre-trained Language Modelscited by 70
  20. 2023Prompting for Multimodal Hateful Meme Classificationcited by 68
  21. 2023ChatGPT is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Modelscited by 63
  22. 2021Prompt-Learning for Fine-Grained Entity Typingcited by 62
  23. 2023GeoLLM: Extracting Geospatial Knowledge from Large Language Modelscited by 62
  24. 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
  25. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 56
  26. 2021A Systematic Investigation of Commonsense Knowledge in Large Language Modelscited by 56
  27. 2021TIMEDIAL: Temporal Commonsense Reasoning in Dialogcited by 55
  28. 2022Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuningcited by 54
  29. 2022Prototypical Verbalizer for Prompt-based Few-shot Tuningcited by 53
  30. 2023Tree-Planner: Efficient Close-loop Task Planning with Large Language Modelscited by 48
  31. 2022Things not Written in Text: Exploring Spatial Commonsense from Visual Signalscited by 41
  32. 2021EventBERT: A Pre-Trained Model for Event Correlation Reasoningcited by 40
  33. 2023ArGue: Attribute-Guided Prompt Tuning for Vision-Language Modelscited by 39
  34. 2021True Few-Shot Learning with Prompts -- A Real-World Perspectivecited by 38
  35. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Surveycited by 38
  36. 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 38
  37. 2022CommunityLM: Probing Partisan Worldviews from Language Modelscited by 36
  38. 2023Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Roboticscited by 35

50 most cited of 189 All 189 (JSON)