Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog
Not on the map. Why?
Abstract (arXiv)
This paper presents a new model for visual dialog, Recurrent Dual Attention Network (ReDAN), using multi-step reasoning to answer a series of questions about an image. In each question-answering turn of a dialog, ReDAN infers the answer progressively through multiple reasoning steps. In each step of the reasoning process, the semantic representation of the question is updated based on the image and the previous dialog history, and the recurrently-refined representation is used for further reasoning in the subsequent step. On the VisDial v1.0 dataset, the proposed ReDAN model achieves a new state-of-the-art of 64.47% NDCG score. Visualization on the reasoning process further demonstrates that ReDAN can locate context-relevant visual and textual clues via iterative refinement, which can lead to the correct answer step-by-step.
Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Feb 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 14; 2022: 19; 2023: 5; 2024: 3.
41 papers.
Sort:
By area:
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
- 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
- 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
- 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversationcited by 56
- 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
- 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
- 2022UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialogcited by 19
- 2021GoG: Relation-aware Graph-over-Graph Network for Visual Dialogcited by 13
- 2021Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generationcited by 11
29 more
- 2021Learning to Ground Visual Objects for Visual Dialogcited by 10
- 2021Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generationcited by 8
- 2024BI-MDRG: Bridging Image History in Multimodal Dialogue Response Generationcited by 8
- 2022The Dialog Must Go On: Improving Visual Dialog via Generative Self-Trainingcited by 7
- 2022VD-PCR: Improving Visual Dialog with Pronoun Coreference Resolutioncited by 7
- 2021Ensemble of MRR and NDCG models for Visual Dialogcited by 5
- 2021Saying the Unseen: Video Descriptions via Dialog Agentscited by 5
- 2021Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guessercited by 5
- 2022Improving Cross-Modal Understanding in Visual Dialog via Contrastive Learningcited by 5
- 2022Unified Multimodal Model with Unlikelihood Training for Visual Dialogcited by 5
- 2023InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Modelscited by 5
- 2022Modeling Coreference Relations in Visual Dialogcited by 3
- 2022Neuro-Symbolic Visual Dialogcited by 3
- 2023FaceChat: An Emotion-Aware Face-to-face Dialogue Frameworkcited by 3
- 2021The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialoguescited by 2
- 2022Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Imagescited by 2
- 2022Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Reviewcited by 2
- 2023A Unified Framework for Slot based Response Generation in a Multimodal Dialogue Systemcited by 1
- 2021SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysiscited by 0
- 2021VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Searchcited by 0
- 2021Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generationcited by 0
- 2022Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectorscited by 0
- 2022Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scenecited by 0
- 2022Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networkscited by 0
- 2022Explanation-Guided Fairness Testing through Genetic Algorithmcited by 0
- 2022Using Graph Neural Networks for Program Terminationcited by 0
- 2023Tribe or Not? Critical Inspection of Group Differences Using TribalGramcited by 0
- 2023VDialogUE: A Unified Evaluation Benchmark for Visually-grounded Dialoguecited by 0
- 2024Enhancing Visual Dialog State Tracking through Iterative Object-Entity Alignment in Multi-Round Conversationscited by 0
- 2024Enhancing Visual Dialog State Tracking through Iterative Object-Entity Alignment in Multi-Round Conversations13 Aug 2024
- 2024BI-MDRG: Bridging Image History in Multimodal Dialogue Response Generation12 Aug 2024
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Video17 Jun 2024
- 2023InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models21 Dec 2023
- 2023VDialogUE: A Unified Evaluation Benchmark for Visually-grounded Dialogue14 Sep 2023
- 2023A Unified Framework for Slot based Response Generation in a Multimodal Dialogue System27 May 2023
- 2023Tribe or Not? Critical Inspection of Group Differences Using TribalGram16 Mar 2023
- 2023FaceChat: An Emotion-Aware Face-to-face Dialogue Framework8 Mar 2023
- 2022Unified Multimodal Model with Unlikelihood Training for Visual Dialog23 Nov 2022
- 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation10 Nov 2022
- 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trends17 Oct 2022
- 2022Vision+X: A Survey on Multimodal Learning in the Light of Data5 Oct 2022
29 more
- 2022Neuro-Symbolic Visual Dialog22 Aug 2022
- 2022Using Graph Neural Networks for Program Termination28 Jul 2022
- 2022Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review2 Jul 2022
- 2022Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Images6 Jun 2022
- 2022VD-PCR: Improving Visual Dialog with Pronoun Coreference Resolution29 May 2022
- 2022The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training25 May 2022
- 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUI23 May 2022
- 2022Explanation-Guided Fairness Testing through Genetic Algorithm16 May 2022
- 2022UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialog1 May 2022
- 2022Improving Cross-Modal Understanding in Visual Dialog via Contrastive Learning15 Apr 2022
- 2022Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene16 Mar 2022
- 2022Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks8 Mar 2022
- 2022Modeling Coreference Relations in Visual Dialog6 Mar 2022
- 2022Discovering Personalized Semantics for Soft Attributes in Recommender Systems using Concept Activation Vectors6 Feb 2022
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan 2022
- 2021Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation17 Sep 2021
- 2021GoG: Relation-aware Graph-over-Graph Network for Visual Dialog17 Sep 2021
- 2021Learning to Ground Visual Objects for Visual Dialog13 Sep 2021
- 2021Text is NOT Enough: Integrating Visual Impressions into Open-domain Dialogue Generation13 Sep 2021
- 2021Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser6 Sep 2021
- 2021VisualTextRank: Unsupervised Graph-based Content Extraction for Automating Ad Text to Image Search5 Aug 2021
- 2021SINGA-Easy: An Easy-to-Use Framework for MultiModal Analysis3 Aug 2021
- 2021Saying the Unseen: Video Descriptions via Dialog Agents26 Jun 2021
- 2021Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generation30 May 2021
- 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey10 May 2021
- 2021Ensemble of MRR and NDCG models for Visual Dialog15 Apr 2021
- 2021Attention, please! A survey of Neural Attention Models in Deep Learning31 Mar 2021
- 2021The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues20 Mar 2021
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm15 Feb 2021
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Videocited by 216
- 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
- 2022META-GUI: Towards Multi-modal Conversational Agents on Mobile GUIcited by 81
- 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversationcited by 56
- 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
- 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
- 2022UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialogcited by 19
- 2021GoG: Relation-aware Graph-over-Graph Network for Visual Dialogcited by 13
- 2021Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generationcited by 11
- 2021Learning to Ground Visual Objects for Visual Dialogcited by 10
- 2021Modeling Text-visual Mutual Dependency for Multi-modal Dialog Generationcited by 8
12 most cited of 39 All 39 (JSON)
- 2024Enhancing Visual Dialog State Tracking through Iterative Object-Entity Alignment in Multi-Round Conversations13 Aug 2024
- 2024BI-MDRG: Bridging Image History in Multimodal Dialogue Response Generation12 Aug 2024
- 2024VideoLLM-online: Online Video Large Language Model for Streaming Video17 Jun 2024
- 2023InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models21 Dec 2023
- 2023VDialogUE: A Unified Evaluation Benchmark for Visually-grounded Dialogue14 Sep 2023
- 2023A Unified Framework for Slot based Response Generation in a Multimodal Dialogue System27 May 2023
- 2023Tribe or Not? Critical Inspection of Group Differences Using TribalGram16 Mar 2023
- 2023FaceChat: An Emotion-Aware Face-to-face Dialogue Framework8 Mar 2023
- 2022Unified Multimodal Model with Unlikelihood Training for Visual Dialog23 Nov 2022
- 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversation10 Nov 2022
- 2022Vision+X: A Survey on Multimodal Learning in the Light of Data5 Oct 2022
- 2022Neuro-Symbolic Visual Dialog22 Aug 2022
12 newest of 39 All 39 (JSON)