mlsanityData to 7 Oct 2026

Transformer-based Localization from Embodied Dialog with Large-scale Pre-training

arXiv 2210.04864 ↗First posted 10 Oct 2022cs.CV cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

We address the challenging task of Localization via Embodied Dialog (LED). Given a dialog from two agents, an Observer navigating through an unknown environment and a Locator who is attempting to identify the Observer's location, the goal is to predict the Observer's final location in a map. We develop a novel LED-Bert architecture and present an effective pretraining strategy. We show that a graph-based scene representation is more effective than the top-down 2D maps used in prior works. Our approach outperforms previous baselines.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 20 of its 26 references that are in the map. First posted on arXiv 10 Oct 2022.

Builds on

20 of 26 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  7. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  9. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  10. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  11. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  12. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
8 more
  1. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  2. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  3. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  4. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  5. 2018Stacked Cross Attention for Image-Text Matchingcited by 414
  6. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  7. 2019Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baselinecited by 60
  8. 2020Where Are You? Localization from Embodied Dialogcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 1; 2026: 1.

5 papers.

  1. 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasonerscited by 48
  2. 2024DiaLoc: An Iterative Approach to Embodied Dialog Localizationcited by 4
  3. 2023Which way is `right'?: Uncovering limitations of Vision-and-Language Navigation modelcited by 3
  4. 2025DialNav: Multi-turn Dialog Navigation with a Remote Guidecited by 3
  5. 2026Advancing DialNav through Automatic Embodied Dialog Augmentationcited by 0