# Online Reinforcement Learning in the Met Office Unified Model through Distributed Model-Agent Coupling

arXiv: https://arxiv.org/abs/2609.02566 · HTML: https://mlsanity.com/paper/2609.02566/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Builds on (most cited first, 7 of 7)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural General Circulation Models for Weather and Climate](https://mlsanity.com/paper/2311.07222/) (2023)
- [Deep learning to represent sub-grid processes in climate models](https://mlsanity.com/paper/1806.04731/) (2018)
- [Earth System Foundation Model (ESFM): A unified framework for heterogeneous data integration and forecasting](https://mlsanity.com/paper/2605.00850/) (2026)
- [Using Machine Learning at Scale in HPC Simulations with SmartSim: An Application to Ocean Climate Modeling](https://mlsanity.com/paper/2104.09355/) (2021)
- [Replacing Tunable Parameters in Weather and Climate Models with State-Dependent Functions using Reinforcement Learning](https://mlsanity.com/paper/2601.04268/) (2026)

## Abstract (arXiv)

Machine-learnt corrections can complement numerical weather prediction provided that they operate stably within an evolving numerical model. In this study, we couple the Met Office (UKMO) Unified Model (UM) with distributed reinforcement-learning agents through rank-local tensors. A column-aware deep deterministic policy gradient (DDPG) actor uses local vertical structure together with full-column context to apply bounded corrections to potential temperature and horizontal wind. During training, we perform ten nudged 6-hr 12-min forecasts, with nudging towards the UKMO operational analysis providing an immediate counterfactual target from which the policy learns. The resulting actor is then frozen and applied to a non-nudged forecast without access to analysis inputs or further weight updates. Relative to a matched non-nudged native forecast at +6 h, the corrected forecast reduces global latitude-weighted MAE by 2.85% for Z500, 2.16% for MSLP, 5.16% for T500 and 2.27% for T_1.5textrmm, with an observed 3.57% wall-time overhead compared to native execution. Even though training and inference share the same initialisation, this single-case experiment demonstrates significant promise and feasibility, laying the groundwork for RL-based bias correction and parametrisations within operational systems.