# Optimizing Heat Alert Issuance with Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.14196 · HTML: https://mlsanity.com/paper/2312.14196/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 71 references that are in the map.
- First posted on arXiv 21 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Explainability in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.06693/) (2020)
- [A Review of Off-Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2212.06355/) (2022)
- [Constraints Penalized Q-learning for Safe Offline Reinforcement Learning](https://mlsanity.com/paper/2107.09003/) (2021)
- [A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environments](https://mlsanity.com/paper/2005.10619/) (2020)
- [Contrastive Explanations for Reinforcement Learning in terms of Expected Consequences](https://mlsanity.com/paper/1807.08706/) (2018)
- [Sample-Efficient Reinforcement Learning in the Presence of Exogenous Information](https://mlsanity.com/paper/2206.04282/) (2022)
- [Hindsight Learning for MDPs with Exogenous Inputs](https://mlsanity.com/paper/2207.06272/) (2022)
- [Learning in POMDPs is Sample-Efficient with Hindsight Observability](https://mlsanity.com/paper/2301.13857/) (2023)
- [Reward Estimation for Variance Reduction in Deep Reinforcement Learning](https://mlsanity.com/paper/1805.03359/) (2018)
- [PerSim: Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents via Personalized Simulators](https://mlsanity.com/paper/2102.06961/) (2021)
- [A policy gradient approach for Finite Horizon Constrained Markov Decision Processes](https://mlsanity.com/paper/2210.04527/) (2022)
- [Deep Reinforcement Learning with Spatio-temporal Traffic Forecasting for Data-Driven Base Station Sleep Control](https://mlsanity.com/paper/2101.08391/) (2021)
- [Hyperparameters in Contextual RL are Highly Situational](https://mlsanity.com/paper/2212.10876/) (2022)
- [A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learning](https://mlsanity.com/paper/2205.04023/) (2022)
- [SpaCE: The Spatial Confounding Environment](https://mlsanity.com/paper/2312.00710/) (2023)
- [Assessing the causal effects of a stochastic intervention in time series data: Are heat alerts effective in preventing deaths and hospitalizations?](https://mlsanity.com/paper/2102.10478/) (2021)
- [Machine learning approaches to identify thresholds in a heat-health warning system context](https://mlsanity.com/paper/2110.10625/) (2021)
- [Constraint Sampling Reinforcement Learning: Incorporating Expertise For Faster Learning](https://mlsanity.com/paper/2112.15221/) (2021)
- [(When) Are Contrastive Explanations of Reinforcement Learning Helpful?](https://mlsanity.com/paper/2211.07719/) (2022)

## Abstract (arXiv)

A key strategy in societal adaptation to climate change is using alert systems to prompt preventative action and reduce the adverse health impacts of extreme heat events. This paper implements and evaluates reinforcement learning (RL) as a tool to optimize the effectiveness of such systems. Our contributions are threefold. First, we introduce a new publicly available RL environment enabling the evaluation of the effectiveness of heat alert policies to reduce heat-related hospitalizations. The rewards model is trained from a comprehensive dataset of historical weather, Medicare health records, and socioeconomic/geographic features. We use scalable Bayesian techniques tailored to the low-signal effects and spatial heterogeneity present in the data. The transition model uses real historical weather patterns enriched by a data augmentation mechanism based on climate region similarity. Second, we use this environment to evaluate standard RL algorithms in the context of heat alert issuance. Our analysis shows that policy constraints are needed to improve RL's initially poor performance. Third, a post-hoc contrastive analysis provides insight into scenarios where our modified heat alert-RL policies yield significant gains/losses over the current National Weather Service alert policy in the United States.