Can Language Model Understand Word Semantics as A Chatbot? An Empirical Study of Language Model Internal External Mismatch
Not on the map. Why?
Abstract (arXiv)
Current common interactions with language models is through full inference. This approach may not necessarily align with the model's internal knowledge. Studies show discrepancies between prompts and internal representations. Most focus on sentence understanding. We study the discrepancy of word semantics understanding in internal and external mismatch across Encoder-only, Decoder-only, and Encoder-Decoder pre-trained language models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 33 references that are in the map. First posted on arXiv 21 Sep 2024.
Builds on
19 of 33 references.
References · 19
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2018Deep contextualized word representationscited by 2,285
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2023The Internal State of an LLM Knows When It's Lyingcited by 644
- 2021Revisiting the Calibration of Modern Neural Networkscited by 387
- 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
7 more
- 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 209
- 2023Knowledge Editing for Large Language Models: A Surveycited by 189
- 2021Truthful AI: Developing and governing AI that does not liecited by 109
- 2021Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Promptscited by 64
- 2023Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness?cited by 62
- 2021BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?cited by 56
- 2023Are Language Models Worse than Humans at Following Prompts? It's Complicatedcited by 14
Built on by
Not cited yet.