mlsanityData to 14 Oct 2026

OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts

arXiv 2109.12761 ↗First posted 27 Sep 2021cs.CLcited by 16

Not on the map. Why?

Abstract (arXiv)

In order to better simulate the real human conversation process, models need to generate dialogue utterances based on not only preceding textual contexts but also visual contexts. However, with the development of multi-modal dialogue learning, the dataset scale gradually becomes a bottleneck. In this report, we release OpenViDial 2.0, a larger-scale open-domain multi-modal dialogue dataset compared to the previous version OpenViDial 1.0. OpenViDial 2.0 contains a total number of 5.6 million dialogue turns extracted from either movies or TV series from different resources, and each dialogue turn is paired with its corresponding visual context. We hope this large-scale dataset can help facilitate future researches on open-domain multi-modal dialog generation, e.g., multi-modal pretraining for dialogue generation.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 56 of its 75 references that are in the map. First posted on arXiv 27 Sep 2021.

Builds on

56 of 75 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  6. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  7. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  8. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  9. 2020Recipes for building an open-domain chatbotcited by 593
  10. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  11. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  12. 2016A Persona-Based Neural Conversation Modelcited by 223
38 more
  1. 2017Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoderscited by 222
  2. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 209
  3. 2015A Neural Conversational Modelcited by 196
  4. 2016A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialoguescited by 174
  5. 2015The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systemscited by 172
  6. 2018The Design and Implementation of XiaoIce, an Empathetic Social Chatbotcited by 172
  7. 2017Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memorycited by 167
  8. 2018Neural Approaches to Conversational AIcited by 165
  9. 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
  10. 2020Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functionscited by 144
  11. 2017A Knowledge-Grounded Neural Conversation Modelcited by 136
  12. 2019Audio-Visual Scene-Aware Dialogcited by 123
  13. 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 113
  14. 2020Knowledge-Grounded Dialogue Generation with Pre-trained Language Modelscited by 107
  15. 2017Adversarial Learning for Neural Dialogue Generationcited by 103
  16. 2015A Neural Network Approach to Context-Sensitive Generation of Conversational Responsescited by 98
  17. 2017Data Noising as Smoothing in Neural Network Language Modelscited by 78
  18. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 71
  19. 2018Visual Coreference Resolution in Visual Dialog using Neural Module Networkscited by 54
  20. 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
  21. 2019CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialogcited by 44
  22. 2017Generating High-Quality and Informative Conversation Responses with Sequence-to-Sequence Modelscited by 42
  23. 2019Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systemscited by 42
  24. 2015Evaluating Prerequisite Qualities for Learning End-to-End Dialog Systemscited by 34
  25. 2018Recursive Visual Attention in Visual Dialogcited by 34
  26. 2018Generating More Interesting Responses in Neural Conversation Models with Distributional Constraintscited by 33
  27. 2019ConvLab: Multi-Domain End-to-End Dialog System Platformcited by 29
  28. 2017Visual Reference Resolution using Attention Memory for Visual Dialogcited by 28
  29. 2018Discourse-Aware Neural Rewards for Coherent Text Generationcited by 28
  30. 2019Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoningcited by 28
  31. 2020OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contextscited by 27
  32. 2019Dual Attention Networks for Visual Reference Resolution in Visual Dialogcited by 25
  33. 2019Reasoning Visual Dialogs with Structural and Partial Observationscited by 25
  34. 2017Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Modelcited by 19
  35. 2017Augmenting End-to-End Dialog Systems with Commonsense Knowledgecited by 18
  36. 2020DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialoguecited by 18
  37. 2017Affective Neural Response Generationcited by 17
  38. 2017Are You Talking to Me? Reasoned Visual Dialog Generation through Adversarial Learningcited by 14

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 4; 2025: 3; 2026: 1.

16 papers.

Sort:

  1. 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversationcited by 56
  2. 2023Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Modelscited by 28
  3. 2022DialogCC: An Automated Pipeline for Creating High-Quality Multi-Modal Dialogue Datasetcited by 17
  4. 2023CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videoscited by 14
  5. 2023VSTAR: A Video-grounded Dialogue Dataset for Situated Semantic Understanding with Scene and Topic Transitionscited by 11
  6. 2024Friends-MMC: A Dataset for Multi-modal Multi-party Conversation Understandingcited by 9
  7. 2024Stark: Social Long-Term Multi-Modal Conversation with Persona Commonsense Knowledgecited by 8
  8. 2024BI-MDRG: Bridging Image History in Multimodal Dialogue Response Generationcited by 8
  9. 2023TikTalk: A Video-Based Dialogue Dataset for Multi-Modal Chitchat in Real Worldcited by 7
  10. 2024A Survey on Multi-modal Machine Translation: Tasks, Methods and Challengescited by 6
  11. 2023Large Language Models can Share Images, Too!cited by 5
  12. 2025Enabling Chatbots with Eyes and Ears: An Immersive Multimodal Conversation System for Dynamic Interactionscited by 3
4 more
  1. 2025KwaiChat: A Large-Scale Video-Driven Multilingual Mixed-Type Dialogue Corpuscited by 1
  2. 2023EXMODD: An EXplanatory Multimodal Open-Domain Dialogue datasetcited by 0
  3. 2025MTPChat: A Multimodal Time-Aware Persona Dataset for Conversational Agentscited by 0
  4. 2026EmotionDialogCN: A Spontaneous Multimodal Dataset for Mandarin Emotional Dialoguecited by 0