mlsanityData to 7 Oct 2026

'No' Matters: Out-of-Distribution Detection in Multimodality Long Dialogue

arXiv 2410.23883 ↗First posted 31 Oct 2024cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Out-of-distribution (OOD) detection in multimodal contexts is essential for identifying deviations in combined inputs from different modalities, particularly in applications like open-domain dialogue systems or real-life dialogue interactions. This paper aims to improve the user experience that involves multi-round long dialogues by efficiently detecting OOD dialogues and images. We introduce a novel scoring framework named Dialogue Image Aligning and Enhancing Framework (DIAEF) that integrates the visual language models with the novel proposed scores that detect OOD in two key scenarios (1) mismatches between the dialogue and image input pair and (2) input pairs with previously unseen labels. Our experimental results, derived from various benchmarks, demonstrate that integrating image and multi-round dialogue OOD detection is more effective with previously unseen labels than using either modality independently. In the presence of mismatched pairs, our proposed score effectively identifies these mismatches and demonstrates strong robustness in long dialogues. This approach enhances domain-aware, adaptive conversational agents and establishes baselines for future studies.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 44 of its 64 references that are in the map. First posted on arXiv 31 Oct 2024.

Builds on

44 of 64 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  6. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  7. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  8. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  9. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  10. 2016Visual Dialogcited by 377
  11. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  12. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
32 more
  1. 2017ParlAI: A Dialog Research Software Platformcited by 139
  2. 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 122
  3. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  4. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  5. 2021Can multi-label classification networks know what they don't know?cited by 100
  6. 2022Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalizationcited by 97
  7. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  8. 2023Generative models improve fairness of medical classifiers under distribution shiftscited by 90
  9. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  10. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  11. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  12. 2014CNN: Single-label to Multi-labelcited by 62
  13. 2019Out-of-domain Detection for Natural Language Understanding in Dialog Systemscited by 62
  14. 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversationcited by 56
  15. 2019CLEVR-Dialog: A Diagnostic Dataset for Multi-Round Reasoning in Visual Dialogcited by 43
  16. 2023Learning to Augment Distributions for Out-of-Distribution Detectioncited by 38
  17. 2023Out-of-distribution Detection Learning with Unreliable Out-of-distribution Sourcescited by 31
  18. 2017Visual Reference Resolution using Attention Memory for Visual Dialogcited by 28
  19. 2021Constructing Multi-Modal Dialogue Dataset by Replacing Text with Semantically Relevant Imagescited by 26
  20. 2021Introspective Distillation for Robust Question Answeringcited by 25
  21. 2022Out-of-distribution Detection via Frequency-regularized Generative Modelscited by 23
  22. 2019What You See is What You Get: Visual Pronoun Coreference Resolution in Dialoguescited by 18
  23. 2022Rethinking Data Augmentation for Robust Visual Question Answeringcited by 18
  24. 2023Exploring Large Language Models for Multi-Modal Out-of-Distribution Detectioncited by 12
  25. 2022On the Generalization for Transfer Learning: An Information-Theoretic Analysiscited by 11
  26. 2021Encoder Adaptation of Dense Passage Retrieval for Open-Domain Question Answeringcited by 8
  27. 2023Improving Open-Set Semi-Supervised Learning with Self-Supervisioncited by 6
  28. 2023Out-of-Domain Intent Detection Considering Multi-Turn Dialogue Contextscited by 6
  29. 2024Interaction Matters: An Evaluation Framework for Interactive Dialogue Assessment on English Second Language Conversationscited by 4
  30. 2021A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithmscited by 3
  31. 2024Multi-Label Out-of-Distribution Detection with Spectral Normalized Joint Energycited by 2
  32. 2024An Interpretable and Crosslingual Method for Evaluating Second-Language Dialoguescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision Also You Need: Navigating Out-of-Distribution Detection with Multimodal Large Language Model
  2. 2024Concept Matching with Agent for Out-of-Distribution Detection
  3. 2026T-QPM: Enabling Temporal Out-Of-Distribution Detection and Domain Generalization for Vision-Language Models in Open-World
  4. 2022Delving into Out-of-Distribution Detection with Vision-Language Representations
  5. 2024Leveraging Chat-Based Large Vision Language Models for Multimodal Out-Of-Context Detection
  6. 2025COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts
  7. 2025Mitigating the Modality Gap: Few-Shot Out-of-Distribution Detection with Multi-modal Prototypes and Image Bias Estimation
  8. 2025Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal Representations