mlsanityData to 7 Oct 2026

Do Neural Language Representations Learn Physical Commonsense?

arXiv 1908.02899 ↗First posted 8 Aug 2019cs.CLcited by 62

Not on the map. Why?

Abstract (arXiv)

Humans understand language based on the rich background knowledge about how the physical world works, which in turn allows us to reason about the physical world through language. In addition to the properties of objects (e.g., boats require fuel) and their affordances, i.e., the actions that are applicable to them (e.g., boats can be driven), we can also reason about if-then inferences between what properties of objects imply the kind of actions that are applicable to them (e.g., that if we can drive something then it likely requires fuel). In this paper, we investigate the extent to which state-of-the-art neural language representations, trained on a vast amount of natural language text, demonstrate physical commonsense reasoning. While recent advancements of neural language models have demonstrated strong performance on various types of natural language inference tasks, our study based on a dataset of over 200k newly collected annotations suggests that neural language representations still only learn associations that are explicitly written down.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 17; 2023: 15; 2024: 5; 2025: 4; 2026: 3.

62 papers.

Sort:

By area:

  1. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  2. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  3. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  4. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  5. 2023Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Imagescited by 79
  6. 2021BERTnesia: Investigating the capture and forgetting of knowledge in BERTcited by 40
  7. 2022Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Surveycited by 38
  8. 2021Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schemacited by 32
  9. 2023Beyond Surface Statistics: Scene Representations in a Latent Diffusion Modelcited by 29
  10. 2021Does Vision-and-Language Pretraining Improve Lexical Grounding?cited by 25
  11. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23
  12. 2023IRFL: Image Recognition of Figurative Languagecited by 23
38 more
  1. 2022VASR: Visual Analogies of Situation Recognitioncited by 22
  2. 2022WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Modelscited by 20
  3. 2021Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understandingcited by 19
  4. 2023Is BERT Blind? Exploring the Effect of Vision-and-Language Pretraining on Visual Language Understandingcited by 19
  5. 2023Can Language Models Understand Physical Concepts?cited by 19
  6. 2021Do Language Models Know the Way to Rome?cited by 18
  7. 2023Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Modelscited by 18
  8. 2022PACS: A Dataset for Physical Audiovisual CommonSense Reasoningcited by 17
  9. 2022Acquiring and Modelling Abstract Commonsense Knowledge via Conceptualizationcited by 17
  10. 2021Distilling Relation Embeddings from Pre-trained Language Modelscited by 15
  11. 2022COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Modelscited by 15
  12. 2023Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 13
  13. 2024Does ChatGPT Have a Mind?cited by 13
  14. 2025Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scalecited by 13
  15. 2021ALL Dolphins Are Intelligent and SOME Are Friendly: Probing BERT for Nouns' Semantic Properties and their Prototypicalitycited by 12
  16. 2021Probing Pre-Trained Language Models for Disease Knowledgecited by 10
  17. 2021Language Models for Lexical Inference in Contextcited by 9
  18. 2022Modelling Commonsense Properties using Pre-Trained Bi-Encoderscited by 9
  19. 2021Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledgecited by 8
  20. 2022A Property Induction Framework for Neural Language Modelscited by 8
  21. 2023Distilling Semantic Concept Embeddings from Contrastively Fine-Tuned Language Modelscited by 8
  22. 2021Open Relation Modeling: Learning to Define Relations between Entitiescited by 7
  23. 2021Language Models as a Knowledge Source for Cognitive Agentscited by 7
  24. 2022Evaluating the Construct Validity of Text Embeddings with Application to Survey Questionscited by 7
  25. 2024On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probecited by 6
  26. 2021Analyzing COVID-19 Tweets with Transformer-based Language Modelscited by 5
  27. 2022What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structuredcited by 5
  28. 2022VIPHY: Probing "Visible" Physical Commonsense Knowledgecited by 4
  29. 2023Probing Physical Reasoning with Counter-Commonsense Contextcited by 4
  30. 2023Learning the Effects of Physical Actions in a Multi-modal Environmentcited by 3
  31. 2023Cabbage Sweeter than Cake? Analysing the Potential of Large Language Models for Learning Conceptual Spacescited by 3
  32. 2023Divergences between Language Models and Human Brainscited by 3
  33. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 3
  34. 2025Exploring Multimodal Perception in Large Language Models Through Perceptual Strength Ratingscited by 3
  35. 2025Seeing What Tastes Good: Revisiting Multimodal Distributional Semantics in the Billion Parameter Eracited by 3
  36. 2021Deriving Word Vectors from Contextualized Language Models using Topic-Aware Mention Selectioncited by 2
  37. 2022On Reality and the Limits of Language Data: Aligning LLMs with Human Normscited by 2
  38. 2022DECK: Behavioral Tests to Improve Interpretability and Generalizability of BERT Models Detecting Depression from Textcited by 2

50 most cited of 62 All 62 (JSON)