mlsanityData to 7 Oct 2026

"No, they did not": Dialogue response dynamics in pre-trained language models

arXiv 2210.02526 ↗First posted 5 Oct 2022cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

A critical component of competence in language is being able to identify relevant components of an utterance and reply appropriately. In this paper we examine the extent of such dialogue response sensitivity in pre-trained language models, conducting a series of experiments with a particular focus on sensitivity to dynamics involving phenomena of at-issueness and ellipsis. We find that models show clear sensitivity to a distinctive role of embedded clauses, and a general preference for responses that target main clause content of prior utterances. However, the results indicate mixed and generally weak trends with respect to capturing the full range of dynamics involved in targeting at-issue versus not-at-issue content. Additionally, models show fundamental limitations in grasp of the dynamics governing ellipsis, and response selections show clear interference from superficial factors that outweigh the influence of principled discourse constraints.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

29 of 45 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  7. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  8. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  9. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  10. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 350
  11. 2018SentEval: An Evaluation Toolkit for Universal Sentence Representationscited by 294
  12. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
17 more
  1. 2019Assessing BERT's Syntactic Abilitiescited by 191
  2. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  3. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  4. 2018Dissecting Contextual Word Embeddings: Architecture and Representationcited by 146
  5. 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
  6. 2019ConveRT: Efficient and Accurate Conversational Representations from Transformerscited by 95
  7. 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
  8. 2021Topic-Driven and Knowledge-Aware Transformer for Dialogue Emotion Detectioncited by 56
  9. 2019Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systemscited by 52
  10. 2016SimVerb-3500: A Large-Scale Evaluation Set of Verb Similaritycited by 44
  11. 2019Probing What Different NLP Tasks Teach Machines about Function Word Comprehensioncited by 44
  12. 2021Discourse Probing of Pretrained Language Modelscited by 38
  13. 2020DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterancescited by 36
  14. 2018Assessing Composition in Sentence Vector Representationscited by 26
  15. 2021Pragmatic competence of pre-trained language models through the lens of discourse connectivescited by 16
  16. 2021Probing Multilingual Language Models for Discoursecited by 12
  17. 2019Does BERT agree? Evaluating knowledge of structure dependence through agreement relationscited by 9

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Language Model Behavior: A Comprehensive Surveycited by 92