# PACT: Peak-Aware Cross-Attention Graph Transformers for Efficient Storm-Surge Emulation

arXiv: https://arxiv.org/abs/2605.09036 · HTML: https://mlsanity.com/paper/2605.09036/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 83 references that are in the map.
- First posted on arXiv 9 May 2026.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Revisiting Single-gated Mixtures of Experts](https://mlsanity.com/paper/2304.05497/) (2023)
- [Recurrent Graph Convolutional Networks for Spatiotemporal Prediction of Snow Accumulation Using Airborne Radar](https://mlsanity.com/paper/2302.00817/) (2023)
- [Prediction of Annual Snow Accumulation Using a Recurrent Graph Convolutional Approach](https://mlsanity.com/paper/2306.13181/) (2023)
- [Prediction of Deep Ice Layer Thickness Using Adaptive Recurrent Graph Neural Networks](https://mlsanity.com/paper/2306.13690/) (2023)
- [An advanced spatio-temporal convolutional recurrent neural network for storm surge predictions](https://mlsanity.com/paper/2204.09501/) (2022)
- [A Framework for Flexible Peak Storm Surge Prediction](https://mlsanity.com/paper/2204.13168/) (2022)
- [Graph convolutional network as a fast statistical emulator for numerical ice sheet modeling](https://mlsanity.com/paper/2402.05291/) (2024)
- [Learning Spatio-Temporal Patterns of Polar Ice Layers With Physics-Informed Graph Neural Network](https://mlsanity.com/paper/2406.15299/) (2024)
- [Multi-branch Spatio-Temporal Graph Neural Network For Efficient Ice Layer Thickness Prediction](https://mlsanity.com/paper/2411.04055/) (2024)
- [ST-GRIT: Spatio-Temporal Graph Transformer For Internal Ice Layer Thickness Prediction](https://mlsanity.com/paper/2507.07389/) (2025)
- [GRIT: Graph Transformer For Internal Ice Layer Thickness Prediction](https://mlsanity.com/paper/2507.07388/) (2025)
- [KAN-GCN: Combining Kolmogorov-Arnold Network with Graph Convolution Network for an Accurate Ice Sheet Emulator](https://mlsanity.com/paper/2510.24926/) (2025)
- [K-STEMIT: Knowledge-Informed Spatio-Temporal Efficient Multi-Branch Graph Neural Network for Subsurface Stratigraphy Thickness Estimation from Radar Data](https://mlsanity.com/paper/2604.09922/) (2026)
- [GRIT-LP: Graph Transformer with Long-Range Skip Connection and Partitioned Spatial Graphs for Accurate Ice Layer Thickness Prediction](https://mlsanity.com/paper/2511.18716/) (2025)
- [StormNet: Improving storm surge predictions with a GNN-based spatio-temporal offset forecasting model](https://mlsanity.com/paper/2604.20688/) (2026)

## Built on by (most cited first, 1 of 1)

- [COGENT: Continuous Graph Emulators with Neural Ordinary Differential Equations for Long-Term Physical Forecasting](https://mlsanity.com/paper/2606.11162/) (2026)

## Abstract (arXiv)

Accurate and efficient storm-surge emulation is essential for coastal hazard assessment, yet high-fidelity hydrodynamic models remain too expensive for large scenario ensembles and rapid evaluation under heterogeneous climate forcings. We present PACT, a peak-aware cross-attention graph transformer for efficient station-level storm-surge prediction from atmospheric forcing fields. PACT represents each forcing patch as a graph, encodes spatial structure with GraphSAGE, and uses a learned station query to aggregate node information through cross-attention rather than uniform pooling. A Transformer encoder models temporal dependence across the forcing history, and a horizon-query decoder generates lead-specific forecasts from a shared temporal memory. To better capture extreme events, we introduce a peak-aware learning strategy that couples a lightweight auxiliary peak-aware head with a tailored training objective, including a tail-focused loss on peak-dominated samples and a horizon-wise slope regularizer to encourage coherent multi-step evolution. Across multiple tide-gauge stations along the US Northeast coast, PACT outperforms a strong spatio-temporal graph neural network baseline in both RMSE and MAE. Diagnostics show improved peak fidelity and tail preservation for reanalysis and most CMIP6 datasets. PACT is also computationally efficient, requiring about 3.5~s to generate a full winter-season surge trajectory for one year after training. Under distribution shift across five CMIP6 forcings, PACT transfers well within the CMIP6 family but degrades markedly when transferring from reanalysis to climate-model forcings, highlighting a persistent reanalysis--GCM gap.