mlsanityData to 14 Oct 2026

Assessing the Limits of the Distributional Hypothesis in Semantic Spaces: Trait-based Relational Knowledge and the Impact of Co-occurrences

arXiv 2205.07603 ↗First posted 16 May 2022cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The increase in performance in NLP due to the prevalence of distributional models and deep learning has brought with it a reciprocal decrease in interpretability. This has spurred a focus on what neural networks learn about natural language with less of a focus on how. Some work has focused on the data used to develop data-driven models, but typically this line of work aims to highlight issues with the data, e.g. highlighting and offsetting harmful biases. This work contributes to the relatively untrodden path of what is required in data for models to capture meaningful representations of natural language. This entails evaluating how well English and Spanish semantic spaces capture a particular type of relational knowledge, namely the traits associated with concepts (e.g. bananas-yellow), and exploring the role of co-occurrences in this context.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 58 references that are in the map. First posted on arXiv 16 May 2022.

Builds on

24 of 58 references.

References · 24

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  3. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  5. 2018Neural Network Acceptability Judgmentscited by 958
  6. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  7. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
  8. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  9. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  10. 2018Targeted Syntactic Evaluation of Language Modelscited by 242
  11. 2018Colorless green recurrent networks dream hierarchicallycited by 213
  12. 2019Assessing BERT's Syntactic Abilitiescited by 192
12 more
  1. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  2. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  3. 2018Under the Hood: Using Diagnostic Classifiers to Investigate and Improve how Language Models Track Agreement Informationcited by 97
  4. 2018What do RNN Language Models Learn about Filler-Gap Dependencies?cited by 86
  5. 2019Inducing Relational Knowledge from BERTcited by 75
  6. 2019Do Neural Language Representations Learn Physical Commonsense?cited by 62
  7. 2018Towards Understanding Linear Word Analogiescited by 57
  8. 2020Investigating Gender Bias in BERTcited by 40
  9. 2018Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Itemscited by 36
  10. 2021The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Colorcited by 32
  11. 2015Take and Took, Gaggle and Goose, Book and Read: Evaluating the Utility of Vector Differences for Lexical Relation Learningcited by 29
  12. 2018RNNs as psycholinguistic subjects: Syntactic state and grammatical dependencycited by 23

Built on by

Not cited yet.