mlsanityData to 14 Oct 2026

Pretraining Methods for Dialog Context Representation Learning

arXiv 1906.00414 ↗First posted 2 Jun 2019cs.CL cs.AIcited by 35

Not on the map. Why?

Abstract (arXiv)

This paper examines various unsupervised pretraining objectives for learning dialog context representations. Two novel methods of pretraining dialog context encoders are proposed, and a total of four methods are examined. Each pretraining objective is fine-tuned and evaluated on a set of downstream dialog tasks using the MultiWoz dataset and strong performance improvement is observed. Further evaluation shows that our pretraining objectives result in not only better performance, but also better convergence, models that are less data hungry and have better domain generalizability.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 11; 2022: 12; 2023: 8; 2024: 1; 2025: 3.

35 papers.

Sort:

By area:

  1. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  2. 2021GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injectioncited by 102
  3. 2021Dialogue State Tracking with a Language Model using Schema-Driven Promptingcited by 94
  4. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  5. 2022InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuningcited by 39
  6. 2022SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generationcited by 34
  7. 2022Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialogcited by 30
  8. 2021ConvFiT: Conversational Fine-Tuning of Pretrained Language Modelscited by 25
  9. 2021Structural Pre-training for Dialogue Comprehensioncited by 20
  10. 2021Effective Sequence-to-Sequence Dialogue State Trackingcited by 20
  11. 2021Cross-lingual Intermediate Fine-tuning improves Dialogue State Trackingcited by 11
  12. 2021Code-switched inspired losses for generic spoken dialog representationscited by 10
23 more
  1. 2021Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generationcited by 8
  2. 2021Representation Learning for Conversational Data using Discourse Mutual Information Maximizationcited by 4
  3. 2022OPAL: Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialoguecited by 4
  4. 2023Pre-training Multi-party Dialogue Models with Latent Discourse Inferencecited by 4
  5. 2024Domain Adaptation in Intent Classification Systems: A Reviewcited by 4
  6. 2022DialAug: Mixing up Dialogue Contexts in Contrastive Learning for Robust Conversational Modelingcited by 3
  7. 2022KPT: Keyword-guided Pre-training for Grounded Dialog Generationcited by 3
  8. 2023Bengali Intent Classification with Generative Adversarial BERTcited by 3
  9. 2021When is it permissible for artificial intelligence to lie? A trust-based approachcited by 2
  10. 2023Matching-based Term Semantics Pre-training for Spoken Patient Query Understandingcited by 2
  11. 2022Learning Locality and Isotropy in Dialogue Modelingcited by 1
  12. 2023An Empirical Study of Multitask Learning to Improve Open Domain Dialogue Systemscited by 1
  13. 2023Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systemscited by 1
  14. 2023Neural Conversation Models and How to Rein Them in: A Survey of Failures and Fixescited by 1
  15. 2025SS-MPC: A Sequence-Structured Multi-Party Conversation Systemcited by 1
  16. 2025Beyond Context: Large Language Models' Failure to Grasp Users' Intentcited by 1
  17. 2022A survey of neural models for the automatic analysis of conversation: Towards a better integration of the social sciencescited by 0
  18. 2022Zero-shot Cross-lingual Conversational Semantic Role Labelingcited by 0
  19. 2022On Task-Adaptive Pretraining for Dialogue Response Selectioncited by 0
  20. 2022SUMBot: Summarizing Context in Open-Domain Dialogue Systemscited by 0
  21. 2022Reranking Overgenerated Responses for End-to-End Task-Oriented Dialogue Systemscited by 0
  22. 2023InterviewBot: Real-Time End-to-End Dialogue System to Interview Students for College Admissioncited by 0
  23. 2025Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandingscited by 0