# EgoArgus: Benchmarking VLMs as Situational Assistants for Modality-Grounded User Supports

arXiv: https://arxiv.org/abs/2608.25561 · HTML: https://mlsanity.com/paper/2608.25561/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 26 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World](https://mlsanity.com/paper/2309.17024/) (2023)
- [OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding?](https://mlsanity.com/paper/2501.05510/) (2025)
- [RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthiness](https://mlsanity.com/paper/2405.17220/) (2024)
- [Grounded Question-Answering in Long Egocentric Videos](https://mlsanity.com/paper/2312.06505/) (2023)
- [Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs](https://mlsanity.com/paper/2510.17771/) (2025)
- [MIntRec: A New Dataset for Multimodal Intent Recognition](https://mlsanity.com/paper/2209.04355/) (2022)
- [Proactive Assistant Dialogue Generation from Streaming Egocentric Videos](https://mlsanity.com/paper/2506.05904/) (2025)
- [Eyes Wide Open: Ego Proactive Video-LLM for Streaming Video](https://mlsanity.com/paper/2510.14560/) (2025)
- [Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization](https://mlsanity.com/paper/2503.17928/) (2025)
- [Mitigating Modal Imbalance in Multimodal Reasoning](https://mlsanity.com/paper/2510.02608/) (2025)
- [Text Takes Over: A Study of Modality Bias in Multimodal Intent Detection](https://mlsanity.com/paper/2508.16122/) (2025)
- [Ego-EXTRA: video-language Egocentric Dataset for EXpert-TRAinee assistance](https://mlsanity.com/paper/2512.13238/) (2025)
- [Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models](https://mlsanity.com/paper/2604.16902/) (2026)
- [VISTA: A Controllable Platform for Generating and Auditing Egocentric Assistance Scenarios](https://mlsanity.com/paper/2605.10579/) (2026)

## Abstract (arXiv)

VLMs are increasingly positioned as daily assistants that perceive first-person environments, follow user dialogue, and decide how to help. Existing egocentric benchmarks mainly evaluate visual understanding in isolation, leaving open whether models can arbitrate between visual evidence and user-provided language when the two are helpful, irrelevant, or conflicting. We introduce EgoArgus, a human-annotated dataset for evaluating egocentric assistants on understanding and decision tasks in five dialogue-video daily scenarios. Our results demonstrate that it is still challenging for current VLMs as reliable egocentric assistants, which requires identifying which modality is trustworthy and deciding when intervention is warranted. Deeper analysis also shows that existing modality bias mitigation methods are quite restricted to enhance performance, providing insights to aid practioners into the deployment of current VLMs as daily assistants.