# Simulating Human Audiovisual Search Behavior

arXiv: https://arxiv.org/abs/2602.02790 · HTML: https://mlsanity.com/paper/2602.02790/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 59 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 9 of 9)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Semantic Audio-Visual Navigation](https://mlsanity.com/paper/2012.11583/) (2020)
- [Modeling Human Exploration Through Resource-Rational Reinforcement Learning](https://mlsanity.com/paper/2201.11817/) (2022)
- [Chartist: Task-driven Eye Movement Control for Chart Reading](https://mlsanity.com/paper/2502.03575/) (2025)
- [A Deep Reinforcement Learning Approach to Audio-Based Navigation in a Multi-Speaker Environment](https://mlsanity.com/paper/2105.04488/) (2021)
- [Auptimize: Optimal Placement of Spatial Audio Cues for Extended Reality](https://mlsanity.com/paper/2408.09320/) (2024)
- [SonoHaptics: An Audio-Haptic Cursor for Gaze-Based Object Selection in XR](https://mlsanity.com/paper/2409.00784/) (2024)
- [Multi-agent Path Finding for Timed Tasks using Evolutionary Games](https://mlsanity.com/paper/2411.10558/) (2024)
- [Audio-Driven Reinforcement Learning for Head-Orientation in Naturalistic Environments](https://mlsanity.com/paper/2409.10048/) (2024)

## Built on by (most cited first, 1 of 1)

- [Can Vision-Language Models Analyze Human-Centered Video? Mapping Model Capabilities and Human-AI Collaborative Workflows](https://mlsanity.com/paper/2609.27327/) (2026)

## Abstract (arXiv)

Locating a target based on auditory and visual cuesunicodex2013such as finding a car in a crowded parking lot or identifying a speaker in a virtual meetingunicodex2013requires balancing effort, time, and accuracy under uncertainty. Existing models of audiovisual search often treat perception and action in isolation, overlooking how people adaptively coordinate movement and sensory strategies. We present Sensonaut, a computational model of embodied audiovisual search. The core assumption is that people deploy their body and sensory systems in ways they believe will most efficiently improve their chances of locating a target, trading off time and effort under perceptual constraints. Our model formulates this as a resource-rational decision-making problem under partial observability. We validate the model against newly collected human data, showing that it reproduces both adaptive scaling of search time and effort under task complexity, occlusion, and distraction, and characteristic human errors. Our simulation of human-like resource-rational search informs the design of audiovisual interfaces that minimize search cost and cognitive load.