Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment
Not on the map. Why?
Abstract (arXiv)
We present an empirical benchmark evaluating how five large language models assess multisensor physical hazard data. Testing 60 scenarios across three categories - multi-sensor joint assessment, response proportionality, and pattern disambiguation - with 1,800 API calls at temperature 0.0, we find that all tested models consistently produced no precautionary warning signal across the tested scenarios where multiple sensors are simultaneously elevated below their individual safety limits, while achieving near-perfect accuracy on single-sensor threshold violations. All five models (ChatGPT-4o, Gemini 2.5 Flash, DeepSeek, Kimi, Llama 3.1 8B) score near zero on Category A multi-sensor scenarios (Q2: 0.000-0.208; Q3: 0.000-0.592) compared to strong performance on single-sensor scenarios (Category B Q1: 0.975-1.000). Structured tabular formatting shows no consistent advantage over plain prose; ChatGPT-4o performs significantly better under prose (p = 0.001). These findings have direct implications for practitioners deploying the tested models in physical safety monitoring systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 15 references that are in the map. First posted on arXiv 25 May 2026.
Builds on
7 of 15 references.
References · 7
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
- 2025HalluLens: LLM Hallucination Benchmarkcited by 92
- 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
- 2024IoT-LLM: a framework for enhancing Large Language Model reasoning from real-world sensor datacited by 18
- 2024Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Modelscited by 12
Built on by
Not cited yet.