# Behavioural gap assessment of human-vehicle interaction in real and virtual reality-based scenarios in autonomous driving

arXiv: https://arxiv.org/abs/2407.04070 · HTML: https://mlsanity.com/paper/2407.04070/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 79 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Builds on (most cited first, 14 of 14)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practice](https://mlsanity.com/paper/1805.11773/) (2018)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [Mind the Gap! A Study on the Transferability of Virtual vs Physical-world Testing of Autonomous Driving Systems](https://mlsanity.com/paper/2112.11255/) (2021)
- [Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real](https://mlsanity.com/paper/2203.11653/) (2022)
- [A Multi-Layered Approach for Measuring the Simulation-to-Reality Gap of Radar Perception for Autonomous Driving](https://mlsanity.com/paper/2106.08372/) (2021)
- [Testing predictive automated driving systems: lessons learned and future recommendations](https://mlsanity.com/paper/2205.10115/) (2022)
- [Realistic pedestrian behaviour in the CARLA simulator using VR and mocap](https://mlsanity.com/paper/2309.04418/) (2023)
- [Insertion of real agents behaviors in CARLA autonomous driving simulator](https://mlsanity.com/paper/2206.00337/) (2022)
- [Digital twin in virtual reality for human-vehicle interactions in the context of autonomous driving](https://mlsanity.com/paper/2303.11463/) (2023)
- [Designing for Projection-based Communication between Autonomous Vehicles and Pedestrians](https://mlsanity.com/paper/2403.06429/) (2024)
- [Human-vehicle interaction for autonomous vehicles in crosswalk scenarios: Field experiments with pedestrians and passengers](https://mlsanity.com/paper/2307.12708/) (2023)
- [To Walk or Not to Walk: Crowdsourced Assessment of External Vehicle-to-Pedestrian Displays](https://mlsanity.com/paper/1707.02698/) (2017)

## Built on by (most cited first, 1 of 1)

- [Cross-cultural analysis of pedestrian group behaviour influence on crossing decisions in interactions with autonomous vehicles](https://mlsanity.com/paper/2408.03003/) (2024)

## Abstract (arXiv)

In the field of autonomous driving research, the use of immersive virtual reality (VR) techniques is widespread to enable a variety of studies under safe and controlled conditions. However, this methodology is only valid and consistent if the conduct of participants in the simulated setting mirrors their actions in an actual environment. In this paper, we present a first and innovative approach to evaluating what we term the behavioural gap, a concept that captures the disparity in a participant's conduct when engaging in a VR experiment compared to an equivalent real-world situation. To this end, we developed a digital twin of a pre-existed crosswalk and carried out a field experiment (N=18) to investigate pedestrian-autonomous vehicle interaction in both real and simulated driving conditions. In the experiment, the pedestrian attempts to cross the road in the presence of different driving styles and an external Human-Machine Interface (eHMI). By combining survey-based and behavioural analysis methodologies, we develop a quantitative approach to empirically assess the behavioural gap, as a mechanism to validate data obtained from real subjects interacting in a simulated VR-based environment. Results show that participants are more cautious and curious in VR, affecting their speed and decisions, and that VR interfaces significantly influence their actions.