Target-Grounded Graph-Aware Transformer for Aerial Vision-and-Dialog Navigation
Not on the map. Why?
Abstract (arXiv)
This report details the methods of the winning entry of the AVDN Challenge in ICCV CLVL 2023. The competition addresses the Aerial Navigation from Dialog History (ANDH) task, which requires a drone agent to associate dialog history with aerial observations to reach the destination. For better cross-modal grounding abilities of the drone agent, we propose a Target-Grounded Graph-Aware Transformer (TG-GAT) framework. Concretely, TG-GAT first leverages a graph-aware transformer to capture spatiotemporal dependency, which benefits navigation state tracking and robust action planning. In addition,an auxiliary visual grounding task is devised to boost the agent's awareness of referred landmarks. Moreover, a hybrid augmentation strategy based on large language models is utilized to mitigate data scarcity limitations. Our TG-GAT framework won the AVDN Challenge, with 2.2% and 3.0% absolute improvements over the baseline on SPL and SR metrics, respectively. The code is available at https://github.com/yifeisu/TG-GAT.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 27 of its 31 references that are in the map. First posted on arXiv 22 Aug 2023.
Builds on
27 of 31 references.
References · 27
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
- 2018Albumentations: fast and flexible image augmentationscited by 560
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
- 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
- 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
15 more
- 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
- 2019Vision-and-Dialog Navigationcited by 250
- 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
- 2021Episodic Transformer for Vision-and-Language Navigationcited by 156
- 2018xView: Objects in Context in Overhead Imagerycited by 146
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
- 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
- 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
- 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
- 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84
- 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
- 2022Aerial Vision-and-Dialog Navigationcited by 49
- 2020Environment-agnostic Multitask Learning for Natural Language Grounded Navigationcited by 43
- 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 43
- 2023CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigationcited by 10
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 1.
2 papers.