Can Visual Dialogue Models Do Scorekeeping? Exploring How Dialogue Representations Incrementally Encode Shared Knowledge
Not on the map. Why?
Abstract (arXiv)
Cognitively plausible visual dialogue models should keep a mental scoreboard of shared established facts in the dialogue context. We propose a theory-based evaluation method for investigating to what degree models pretrained on the VisDial dataset incrementally build representations that appropriately do scorekeeping. Our conclusion is that the ability to make the distinction between shared and privately known statements along the dialogue is moderately present in the analysed models, but not always incrementally consistent, which may partially be due to the limited need for grounding interactions in the original task.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 26 of its 55 references that are in the map. First posted on arXiv 14 Apr 2022.
Builds on
26 of 55 references.
References · 26
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2019Designing and Interpreting Probes with Control Taskscited by 608
- 2016Visual Dialogcited by 377
- 2018Analysis Methods in Neural Language Processing: A Surveycited by 276
- 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
- 2015A Neural Conversational Modelcited by 195
- 2018Higher-order Coreference Resolution with Coarse-to-fine Inferencecited by 150
- 2018Dialogue Natural Language Inferencecited by 135
14 more
- 2018Transforming Question Answering Datasets Into Natural Language Inference Datasetscited by 99
- 2015A Neural Network Approach to Context-Sensitive Generation of Conversational Responsescited by 98
- 2017FOIL it! Find One mismatch between Image and Language captioncited by 95
- 2019Evaluating Coherence in Dialogue Systems using Entailmentcited by 53
- 2019Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Studycited by 47
- 2020History for Visual Dialog: Do we really need it?cited by 32
- 2021Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQAcited by 30
- 2019Improving Generative Visual Dialog by Answering Diverse Questionscited by 21
- 2018Beyond task success: A closer look at jointly learning to see, ask, and GuessWhatcited by 15
- 2020Probing Task-Oriented Dialogue Representation from Language Modelscited by 15
- 2019Grounded Agreement Games: Emphasizing Conversational Grounding in Visual Dialogue Settingscited by 10
- 2020Probing Neural Dialog Models for Conversational Understandingcited by 10
- 2018Analysing the potential of seq-to-seq models for incremental interpretation in task-oriented dialoguecited by 3
- 2020How To Evaluate Your Dialogue System: Probe Tasks as an Alternative for Token-level Evaluation Metricscited by 3
Built on by
Citing papers per year: 2023: 2.
2 papers.