mlsanityData to 7 Oct 2026

Local and Global Contexts for Conversation

arXiv 2401.17588 ↗First posted 31 Jan 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The context in conversation is the dialog history crucial for multi-turn dialogue. Learning from the relevant contexts in dialog history for grounded conversation is a challenging problem. Local context is the most neighbor and more sensitive to the subsequent response, and global context is relevant to a whole conversation far beyond neighboring utterances. Currently, pretrained transformer models for conversation challenge capturing the correlation and connection between local and global contexts. We introduce a local and global conversation model (LGCM) for general-purpose conversation in open domain. It is a local-global hierarchical transformer model that excels at accurately discerning and assimilating the relevant contexts necessary for generating responses. It employs a local encoder to grasp the local context at the level of individual utterances and a global encoder to understand the broader context at the dialogue level. The seamless fusion of these locally and globally contextualized encodings ensures a comprehensive comprehension of the conversation. Experiments on popular datasets show that LGCM outperforms the existing conversation models on the performance of automatic metrics with significant margins.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

34 of 53 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  6. 2018Self-Attention with Relative Position Representationscited by 1,112
  7. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  8. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  9. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  10. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  11. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  12. 2020Recipes for building an open-domain chatbotcited by 593
22 more
  1. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  2. 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
  3. 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
  4. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  5. 2020Lite Transformer with Long-Short Range Attentioncited by 127
  6. 2019HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarizationcited by 120
  7. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
  8. 2020Improve Transformer Models with Better Relative Position Embeddingscited by 73
  9. 2015A Hierarchical Recurrent Encoder-Decoder For Generative Context-Aware Query Suggestioncited by 68
  10. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  11. 2019Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Studycited by 47
  12. 2022Hierarchical Local-Global Transformer for Temporal Sentence Groundingcited by 38
  13. 2020DialogBERT: Discourse-Aware Response Generation via Learning to Recover and Rank Utterancescited by 36
  14. 2017Hierarchical Recurrent Attention Network for Response Generationcited by 34
  15. 2018SDNet: Contextualized Attention-based Deep Network for Conversational Question Answeringcited by 30
  16. 2016Multiresolution Recurrent Neural Networks: An Application to Dialogue Response Generationcited by 16
  17. 2020Hierarchical Transformer for Task Oriented Dialog Systemscited by 14
  18. 2020DialBERT: A Hierarchical Pre-Trained Model for Conversation Disentanglementcited by 12
  19. 2021DialogueBERT: A Self-Supervised Learning based Dialogue Pre-training Encodercited by 11
  20. 2021Jurassic is (almost) All You Need: Few-Shot Meaning-to-Text Generation for Open-Domain Dialoguecited by 6
  21. 2021A Context-Aware Hierarchical BERT Fusion Network for Multi-turn Dialog Act Detectioncited by 5
  22. 2020Which Kind Is Better in Open-domain Multi-turn Dialog,Hierarchical or Non-hierarchical Models? An Empirical Studycited by 3

Built on by

Not cited yet.