mlsanityData to 14 Oct 2026

VU-BERT: A Unified framework for Visual Dialog

arXiv 2202.10787 ↗First posted 22 Feb 2022cs.CL cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The visual dialog task attempts to train an agent to answer multi-turn questions given an image, which requires the deep understanding of interactions between the image and dialog history. Existing researches tend to employ the modality-specific modules to model the interactions, which might be troublesome to use. To fill in this gap, we propose a unified framework for image-text joint embedding, named VU-BERT, and apply patch projection to obtain vision embedding firstly in visual dialog tasks to simplify the model. The model is trained over two tasks: masked language modeling and next utterance retrieval. These tasks help in learning visual concepts, utterances dependence, and the relationships between these two modalities. Finally, our VU-BERT achieves competitive performance (0.7287 NDCG scores) on VisDial v1.0 Datasets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 22 Feb 2022.

Builds on

17 of 20 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  5. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  6. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  7. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  8. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  9. 2016Visual Dialogcited by 377
  10. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  11. 2019Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baselinecited by 61
  12. 2020VD-BERT: A Unified Vision and Dialog Transformer with BERTcited by 55
5 more
  1. 2019Learning Cross-modal Context Graph for Visual Groundingcited by 49
  2. 2019Factor Graph Attentioncited by 41
  3. 2019DualVD: An Adaptive Dual Encoding Model for Deep Visual Understanding in Visual Dialoguecited by 30
  4. 2020Iterative Context-Aware Graph Inference for Visual Dialogcited by 16
  5. 2020Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triagecited by 16

Built on by

Not cited yet.