# CyPortQA: Benchmarking Multimodal Large Language Models for Cyclone Preparedness in Port Operation

arXiv: https://arxiv.org/abs/2508.15846 · HTML: https://mlsanity.com/paper/2508.15846/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 39 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://mlsanity.com/paper/2304.00723/) (2023)
- [DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response](https://mlsanity.com/paper/2505.21089/) (2025)
- [Harnessing Large Language Models for Disaster Management: A Survey](https://mlsanity.com/paper/2501.06932/) (2025)
- [From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMs](https://mlsanity.com/paper/2502.17701/) (2025)
- [Corvid: Improving Multimodal Large Language Models Towards Chain-of-Thought Reasoning](https://mlsanity.com/paper/2507.07424/) (2025)
- [Unleashing the Potential of Large Language Model: Zero-shot VQA for Flood Disaster Scenario](https://mlsanity.com/paper/2312.01882/) (2023)
- [Global Tropical Cyclone Intensity Forecasting with Multi-modal Multi-scale Causal Autoregressive Model](https://mlsanity.com/paper/2402.13270/) (2024)
- [Integration of Large Vision Language Models for Efficient Post-disaster Damage Assessment and Reporting](https://mlsanity.com/paper/2411.01511/) (2024)
- [LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment](https://mlsanity.com/paper/2506.06355/) (2025)
- [Identifying and Investigating Global News Coverage of Critical Events Such as Disasters and Terrorist Attacks](https://mlsanity.com/paper/2506.12925/) (2025)

## Built on by (most cited first, 3 of 3)

- [A Statistical Framework for Auditing Behavioral Dependence and Induced Bias in LLM Judges](https://mlsanity.com/paper/2604.07650/) (2026)
- [Knowledge-Grounded Agentic Large Language Models for Multi-Hazard Understanding from Reconnaissance Reports](https://mlsanity.com/paper/2511.14010/) (2025)
- [Knowledge Is Not Static: Order-Aware Hypergraph RAG for Language Models](https://mlsanity.com/paper/2604.12185/) (2026)

## Abstract (arXiv)

As tropical cyclones intensify and track forecasts become increasingly uncertain, U.S. ports face heightened supply-chain risk under extreme weather conditions. Port operators need to rapidly synthesize diverse multimodal forecast products, such as probabilistic wind maps, track cones, and official advisories, into clear, actionable guidance as cyclones approach. Multimodal large language models (MLLMs) offer a powerful means to integrate these heterogeneous data sources alongside broader contextual knowledge, yet their accuracy and reliability in the specific context of port cyclone preparedness have not been rigorously evaluated. To fill this gap, we introduce CyPortQA, the first multimodal benchmark tailored to port operations under cyclone threat. CyPortQA assembles 2,917 realworld disruption scenarios from 2015 through 2023, spanning 145 U.S. principal ports and 90 named storms. Each scenario fuses multisource data (i.e., tropical cyclone products, port operational impact records, and port condition bulletins) and is expanded through an automated pipeline into 117,178 structured question answer pairs. Using this benchmark, we conduct extensive experiments on diverse MLLMs, including both open-source and proprietary model. MLLMs demonstrate great potential in situation understanding but still face considerable challenges in reasoning tasks, including potential impact estimation and decision reasoning.