# Gaze-based Human-Robot Interaction System for Infrastructure Inspections

arXiv: https://arxiv.org/abs/2403.08061 · HTML: https://mlsanity.com/paper/2403.08061/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2024.

## Built on by (most cited first, 2 of 2)

- [USS: Unifying Spatial-Semantic Prompting for End to End Embodied Visual Tracking](https://mlsanity.com/paper/2606.25880/) (2026)
- [CERBERUS: Crack Evaluation & Recognition Benchmark for Engineering Reliability & Urban Stability](https://mlsanity.com/paper/2506.21909/) (2025)

## Abstract (arXiv)

Routine inspections for critical infrastructures such as bridges are required in most jurisdictions worldwide. Such routine inspections are largely visual in nature, which are qualitative, subjective, and not repeatable. Although robotic infrastructure inspections address such limitations, they cannot replace the superior ability of experts to make decisions in complex situations, thus making human-robot interaction systems a promising technology. This study presents a novel gaze-based human-robot interaction system, designed to augment the visual inspection performance through mixed reality. Through holograms from a mixed reality device, gaze can be utilized effectively to estimate the properties of the defect in real-time. Additionally, inspectors can monitor the inspection progress online, which enhances the speed of the entire inspection process. Limited controlled experiments demonstrate its effectiveness across various users and defect types. To our knowledge, this is the first demonstration of the real-time application of eye gaze in civil infrastructure inspections.