# Commands 4 Autonomous Vehicles (C4AV) Workshop Summary

arXiv: https://arxiv.org/abs/2009.08792 · HTML: https://mlsanity.com/paper/2009.08792/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2020.

## Built on by (most cited first, 5 of 5)

- [GPT-4 Enhanced Multimodal Grounding for Autonomous Driving: Leveraging Cross-Modal Attention with Large Language Models](https://mlsanity.com/paper/2312.03543/) (2023)
- [Giving Commands to a Self-Driving Car: How to Deal with Uncertain Situations?](https://mlsanity.com/paper/2106.04232/) (2021)
- [Talk to Parallel LiDARs: A Human-LiDAR Interaction Method Based on 3D Visual Grounding](https://mlsanity.com/paper/2405.15274/) (2024)
- [Predicting Physical World Destinations for Commands Given to Self-Driving Cars](https://mlsanity.com/paper/2112.05419/) (2021)
- [Estimation of Appearance and Occupancy Information in Birds Eye View from Surround Monocular Images](https://mlsanity.com/paper/2211.04557/) (2022)

## Abstract (arXiv)

The task of visual grounding requires locating the most relevant region or object in an image, given a natural language query. So far, progress on this task was mostly measured on curated datasets, which are not always representative of human spoken language. In this work, we deviate from recent, popular task settings and consider the problem under an autonomous vehicle scenario. In particular, we consider a situation where passengers can give free-form natural language commands to a vehicle which can be associated with an object in the street scene. To stimulate research on this topic, we have organized the Commands for Autonomous Vehicles (C4AV) challenge based on the recent Talk2Car dataset (URL: https://www.aicrowd.com/challenges/eccv-2020-commands-4-autonomous-vehicles). This paper presents the results of the challenge. First, we compare the used benchmark against existing datasets for visual grounding. Second, we identify the aspects that render top-performing models successful, and relate them to existing state-of-the-art models for visual grounding, in addition to detecting potential failure cases by evaluating on carefully selected subsets. Finally, we discuss several possibilities for future work.