# LAVQA: A Latency-Aware Visual Question Answering Framework for Shared Autonomy in Self-Driving Vehicles

arXiv: https://arxiv.org/abs/2511.11840 · HTML: https://mlsanity.com/paper/2511.11840/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 6 of its 41 references that are in the map.
- First posted on arXiv 14 Nov 2025.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 6 of 6)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024)
- [Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasets](https://mlsanity.com/paper/2505.15517/) (2025)
- [Shared Autonomy with Learned Latent Actions](https://mlsanity.com/paper/2005.03210/) (2020)
- [Can Large Vision Language Models Read Maps Like a Human?](https://mlsanity.com/paper/2503.14607/) (2025)
- [What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning](https://mlsanity.com/paper/2205.02671/) (2022)

## Abstract (arXiv)

When uncertainty is high, self-driving vehicles may halt for safety and benefit from the access to remote human operators who can provide high-level guidance. This paradigm, known as {shared autonomy}, enables autonomous vehicle and remote human operators to jointly formulate appropriate responses. To address critical decision timing with variable latency due to wireless network delays and human response time, we present LAVQA, a latency-aware shared autonomy framework that integrates Visual Question Answering (VQA) and spatiotemporal risk visualization. LAVQA augments visual queries with Latency-Induced COllision Map (LICOM), a dynamically evolving map that represents both temporal latency and spatial uncertainty. It enables remote operator to observe as the vehicle safety regions vary over time in the presence of dynamic obstacles and delayed responses. Closed-loop simulations in CARLA, the de-facto standard for autonomous vehicle simulator, suggest that that LAVQA can reduce collision rates by over 8x compared to latency-agnostic baselines.