From Sports to Safety: Benchmarking Proactive Risk Inference in MLLMs
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Timely anticipation of physical hazards is essential for real-world safety, yet existing MLLM evaluations focus on harmful content or general risks, leaving proactive physical hazard prediction underexplored. Sports provide a well-suited testbed: accident causes span diverse injury dimensions and pre-accident spatiotemporal cues draw on reasoning capabilities shared with broader safety domains such as autonomous driving and fall detection. We introduce SPRINT (Sports Proactive Risk INference Testbed), a benchmark of 2,888 real-world sports videos (2,440 accident, 448 safe controls) spanning 14 sports and 3 environmental settings. Accident videos feature fine-grained annotations of early hazard cues, accident timing, and hierarchical causes; safe videos are manually verified as accident-free and serve to diagnose prompt-induced false alarms. Evaluating state-of-the-art MLLMs under diverse prompts and temporal windows reveals a sharp gap between hazard sensitivity and understanding: the best model exceeds 95% in signaling hazards yet falls below 50% in identifying their causes. Diagnostic experiments further show that explicit danger queries trigger severe false alarms even on hazard-free videos. These findings indicate that current MLLMs exhibit only superficial proactive safety, lacking stable, cause-grounded early warning, and underscore the need for reliable proactive safety in dynamic physical environments. Data and code will be open-sourced upon acceptance.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 34 of its 64 references that are in the map. First posted on arXiv 6 Aug 2026.
Builds on
34 of 64 references.
Closest ancestors · 2
Same direction, newest first
- 2024SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations9 Dec
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
Other references · 32
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
- 2022Training language models to follow instructions with human feedbackcited by 15,856
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,064
- 2025Qwen3-VL Technical Reportcited by 2,663
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,637
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,452
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,405
- 2022Red Teaming Language Models with Language Modelscited by 1,014
- 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 853
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 531
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 404
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 292
20 more
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 280
- 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 115
- 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 100
- 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 99
- 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agentscited by 88
- 2024Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistancecited by 85
- 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 73
- 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
- 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 60
- 2019DADA-2000: Can Driving Accident be Predicted by Driver Attention? Analyzed by A Benchmarkcited by 57
- 2024Towards Human-centered Proactive Conversational Agentscited by 48
- 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 44
- 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
- 2023DisasterResponseGPT: Large Language Models for Accelerated Plan of Action Development in Disaster Response Scenarioscited by 34
- 2025Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMscited by 27
- 2024LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaborationcited by 21
- 2024Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysiscited by 9
- 2025Towards Evaluating Proactive Risk Awareness of Multimodal Language Modelscited by 6
- 2025RESPONSE: Benchmarking the Ability of Language Models to Undertake Commonsense Reasoning in Crisis Situationcited by 4
- 2026PaSBench-Video: A Streaming Video Benchmark for Proactive Safety Warningcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025AccidentBench: Benchmarking Multimodal Understanding and Reasoning in Vehicle Accidents and Beyond
- 2024Using Multimodal Large Language Models for Automated Detection of Traffic Safety Critical Events
- 2026Hazard or Anomaly? Evaluating VLMs for Understanding Dangers and Discrepancies
- 2025DRAMA-X: A Fine-grained Intent Prediction and Risk Reasoning Benchmark For Driving
- 2026General Hazard Detection
- 2025iSafetyBench: A video-language benchmark for safety in industrial environment
- 2026CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarios
- 2023Exploring the Potential of Multi-Modal AI for Driving Hazard Prediction