mlsanityData to 7 Oct 2026

Target-dependent UNITER: A Transformer-Based Multimodal Language Comprehension Model for Domestic Service Robots

arXiv 2107.00811 ↗First posted 2 Jul 2021cs.RO cs.CL cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Currently, domestic service robots have an insufficient ability to interact naturally through language. This is because understanding human instructions is complicated by various ambiguities and missing information. In existing methods, the referring expressions that specify the relationships between objects are insufficiently modeled. In this paper, we propose Target-dependent UNITER, which learns the relationship between the target object and other objects directly by focusing on the relevant regions within an image, rather than the whole image. Our method is an extension of the UNITER-based Transformer that can be pretrained on general-purpose datasets. We extend the UNITER approach by introducing a new architecture for handling the target candidates. Our model is validated on two standard datasets, and the results show that Target-dependent UNITER outperforms the baseline method in terms of classification accuracy.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 2 Jul 2021.

Builds on

19 of 29 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  7. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  8. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  9. 2019Visual Entailment: A Novel Task for Fine-Grained Image Understandingcited by 250
  10. 2014A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Inputcited by 112
  11. 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 112
  12. 2018Attention Branch Network: Learning of Attention Mechanism for Visual Explanationcited by 108
7 more
  1. 2019CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressionscited by 73
  2. 2017Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructionscited by 61
  3. 2019Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methodscited by 55
  4. 2017Comprehension-guided referring expressionscited by 52
  5. 2019Understanding Natural Language Instructions for Fetching Daily Objects Using GAN-Based Multimodal Target-Source Classificationcited by 16
  6. 2019A Multimodal Target-Source Classifier with Attention Branches to Understand Ambiguous Instructions for Fetching Daily Objectscited by 6
  7. 2020Alleviating the Burden of Labeling: Sentence Generation by Attention Branch Encoder-Decoder Networkcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 6.

7 papers.

  1. 2022Moment-based Adversarial Training for Embodied Language Comprehensioncited by 8
  2. 2023Multimodal Diffusion Segmentation Model for Object Segmentation from Manipulation Instructionscited by 7
  3. 2023Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Taskscited by 5
  4. 2023Learning-To-Rank Approach for Identifying Everyday Objects Using a Physical-World Search Enginecited by 5
  5. 2023Natural Language Instructions for Intuitive Human Interaction with Robotic Assistants in Field Construction Workcited by 3
  6. 2023Naming Objects for Vision-and-Language Manipulationcited by 1
  7. 2023Prototypical Contrastive Transfer Learning for Multimodal Language Understandingcited by 1