mlsanityData to 7 Oct 2026

Transparency Helps Reveal When Language Models Learn Meaning

arXiv 2210.07468 ↗First posted 14 Oct 2022cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Many current NLP systems are built from language models trained to optimize unsupervised objectives on large amounts of raw text. Under what conditions might such a procedure acquire meaning? Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations (i.e., languages with strong transparency), both autoregressive and masked language models successfully learn to emulate semantic relations between expressions. However, when denotations are changed to be context-dependent with the language otherwise unmodified, this ability degrades. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not represent natural language semantics well. We show this failure relates to the context-dependent nature of natural language form-meaning mappings.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 38 of its 77 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

38 of 77 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 770
  11. 2019Designing and Interpreting Probes with Control Taskscited by 608
  12. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
26 more
  1. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  2. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  3. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
  4. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  5. 2019Revealing the Dark Secrets of BERTcited by 300
  6. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  7. 2020AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Promptscited by 246
  8. 2018Targeted Syntactic Evaluation of Language Modelscited by 239
  9. 2018Colorless green recurrent networks dream hierarchicallycited by 210
  10. 2021Implicit Representations of Meaning in Neural Language Modelscited by 179
  11. 2020Probing Pretrained Language Models for Lexical Semanticscited by 156
  12. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 148
  13. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  14. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  15. 2019E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTcited by 96
  16. 2019Measuring Compositionality in Representation Learningcited by 84
  17. 2020On the Sentence Embeddings from Pre-trained Language Modelscited by 83
  18. 2021Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?cited by 55
  19. 2019Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasetscited by 41
  20. 2020Assessing Phrasal Representation and Composition in Transformerscited by 41
  21. 2021Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Sensescited by 39
  22. 2021Sorting through the noise: Testing robustness of information processing in pre-trained language modelscited by 27
  23. 2020Infusing Finetuning with Semantic Dependenciescited by 18
  24. 2021Static Embeddings as Efficient Knowledge Bases?cited by 11
  25. 2021On the Interplay Between Fine-tuning and Composition in Transformerscited by 6
  26. 2021Is "My Favorite New Movie" My Favorite Movie? Probing the Understanding of Recursive Noun Phrasescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 2; 2026: 2.

8 papers.

  1. 2023Meaning Representations from Trajectories in Autoregressive Modelscited by 18
  2. 2024ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?cited by 17
  3. 2025Token embeddings violate the manifold hypothesiscited by 14
  4. 2024Can You Learn Semantics Through Next-Word Prediction? The Case of Entailmentcited by 9
  5. 2024A critical review of methods and challenges in large language modelscited by 2
  6. 2025Gamma Mixture Modeling for Cosine Similarity in Small Language Modelscited by 0
  7. 2026LLM-Guided Semantic Bootstrapping for Interpretable Text Classification with Tsetlin Machinescited by 0
  8. 2026Clusters are All You Need: Pre-Training the Tsetlin Machine with Semantic Clusters from Language Models for Interpretabilitycited by 0