# Exploiting Natural Language for Efficient Risk-Aware Multi-robot SaR Planning

arXiv: https://arxiv.org/abs/2104.03809 · HTML: https://mlsanity.com/paper/2104.03809/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2021.

## Built on by (most cited first, 8 of 8)

- [CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes](https://mlsanity.com/paper/2206.04382/) (2022)
- [From Screens to Scenes: A Survey of Embodied AI in Healthcare](https://mlsanity.com/paper/2501.07468/) (2025)
- [Robot Navigation in Risky, Crowded Environments: Understanding Human Preferences](https://mlsanity.com/paper/2303.08284/) (2023)
- [Learning to Assess Danger from Movies for Cooperative Escape Planning in Hazardous Environments](https://mlsanity.com/paper/2207.13791/) (2022)
- [Dual-Attention Heterogeneous GNN for Multi-robot Collaborative Area Search via Deep Reinforcement Learning](https://mlsanity.com/paper/2601.03686/) (2026)
- [Absence is Presence: Understanding Visual Scene Negative Events Under Safety Cognitive Constraint](https://mlsanity.com/paper/2609.19812/) (2026)
- [Combining Hierarchical Cognitive Process with Process Supervision for Interpretable Scene Safety Understanding](https://mlsanity.com/paper/2609.26399/) (2026)
- [Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments](https://mlsanity.com/paper/2609.27076/) (2026)

## Abstract (arXiv)

The ability to develop a high-level understanding of a scene, such as perceiving danger levels, can prove valuable in planning multi-robot search and rescue (SaR) missions. In this work, we propose to uniquely leverage natural language descriptions from the mission commander in chief and image data captured by robots to estimate scene danger. Given a description and an image, a state-of-the-art deep neural network is used to assess a corresponding similarity score, which is then converted into a probabilistic distribution of danger levels. Because commonly used visio-linguistic datasets do not represent SaR missions well, we collect a large-scale image-description dataset from synthetic images taken from realistic disaster scenes and use it to train our machine learning model. A risk-aware variant of the Multi-robot Efficient Search Path Planning (MESPP) problem is then formulated to use the danger estimates in order to account for high-risk locations in the environment when planning the searchers' paths. The problem is solved via a distributed approach based on Mixed-Integer Linear Programming. Our experiments demonstrate that our framework allows to plan safer yet highly successful search missions, abiding to the two most important aspects of SaR missions: to ensure both searchers' and victim safety.