# Risk Is Not the Target: A Monotonic Framework for Evaluating Wildfire Operational Risk Signals

arXiv: https://arxiv.org/abs/2607.21597 · HTML: https://mlsanity.com/paper/2607.21597/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 23 Apr 2026.

## Builds on (most cited first, 9 of 9)

- [Learning Optimized Risk Scores](https://mlsanity.com/paper/1610.00168/) (2016)
- [WildfireGPT: Tailored Large Language Model for Wildfire Analysis](https://mlsanity.com/paper/2402.07877/) (2024)
- [Performance evaluation of predictive AI models to support medical decisions: Overview and guidance](https://mlsanity.com/paper/2412.10288/) (2024)
- [Empowering LLM Agents with Geospatial Awareness: Toward Grounded Reasoning for Wildfire Response](https://mlsanity.com/paper/2510.12061/) (2025)
- [FireCastNet: Earth-as-a-Graph for Seasonal Fire Prediction](https://mlsanity.com/paper/2502.01550/) (2025)
- [Agentic LLM Framework for Adaptive Decision Discourse](https://mlsanity.com/paper/2502.10978/) (2025)
- [SeasFire as a Multivariate Earth System Datacube for Wildfire Dynamics](https://mlsanity.com/paper/2312.07199/) (2023)
- [Localized Forest Fire Risk Prediction: A Department-Aware Approach for Operational Decision Support](https://mlsanity.com/paper/2506.04254/) (2025)
- [Proof of Concept: Multi-Target Wildfire Risk Prediction and Large Language Model Synthesis](https://mlsanity.com/paper/2601.11686/) (2026)

## Abstract (arXiv)

Evaluating wildfire risk systems using standard machine-learning metrics such as F1-score or IoU is fundamentally flawed: these metrics assess event prediction accuracy, not the operational coherence of a continuous risk signal. This work proposes a novel monotonic evaluation framework that measures whether increases in a predicted risk score consistently correspond to increases in observed operational load, such as number of fires, intervention time, and deployed resources. Moreover, we compare three structurally different approaches on the French Alpes-Maritimes department: the expert-based DFE index, GRU- based predictive models, and FARS, a hybrid multi-agent system combining predictive AI with LLM-based reasoning. Experimental results reveal that the DFE, despite poor classification metrics, exhibits the most balanced monotonic behavior across the full risk scale. GRU models achieve strong local monotonicity but fail to produce well-distributed risk levels. FARS inherits and reveals the structural limitations of upstream signals rather than correcting them. The central finding is a paradigm shift: a good risk model does not predict fires accurately, but one whose ordinal scale meaningfully explains operational dynamics, as proved in this paper. Code of the monotonic framework is available on github.