# Toward Mechanistic Interpretability of an AI Foundation Model Fine-Tuned for Atmospheric Chemistry

arXiv: https://arxiv.org/abs/2607.20778 · HTML: https://mlsanity.com/paper/2607.20778/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 62 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [Surgical Fine-Tuning Improves Adaptation to Distribution Shifts](https://mlsanity.com/paper/2210.11466/) (2022)
- [Improving Steering Vectors by Targeting Sparse Autoencoder Features](https://mlsanity.com/paper/2411.02193/) (2024)
- [Rethinking the Hyperparameters for Fine-tuning](https://mlsanity.com/paper/2002.11770/) (2020)
- [Physically Interpretable Neural Networks for the Geosciences: Applications to Earth System Variability](https://mlsanity.com/paper/1912.01752/) (2019)
- [Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science](https://mlsanity.com/paper/2303.00652/) (2023)
- [Causal Language Control in Multilingual Transformers via Sparse Feature Steering](https://mlsanity.com/paper/2507.13410/) (2025)
- [Towards mechanistic understanding in a data-driven weather model: internal activations reveal interpretable physical features](https://mlsanity.com/paper/2512.24440/) (2025)
- [No One Knows the State of the Art in Geospatial Foundation Models](https://mlsanity.com/paper/2605.12678/) (2026)
- [PhysMetrics.Weather: An Evaluation Framework for Physical Consistency in ML Weather Models](https://mlsanity.com/paper/2606.10642/) (2026)
- [Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs](https://mlsanity.com/paper/2605.21602/) (2026)
- [Orders-of-magnitude speedup in atmospheric chemistry modeling through neural network-based emulation](https://mlsanity.com/paper/1808.03874/) (2018)

## Built on by (most cited first, 1 of 1)

- [Strategic Governance of AI Models in Earth Science](https://mlsanity.com/paper/2610.10560/) (2026)

## Abstract (arXiv)

Weather forecasting foundation models (FMs) are increasingly fine-tuned to predict air quality, offering fast global pollution forecasts at lower computational cost than conventional chemical transport models. These FMs are typically trained on reanalysis data and generate forecasts through autoregressive rollout. They do not explicitly represent governing physical or chemical processes. Therefore, high forecast skill does not reveal whether a model has learned physical mechanisms or exploits statistical regularities in its training data. Here, we present the first study of what a FM fine-tuned for atmospheric chemistry has learned by examining Microsoft's Aurora model. We impose controlled chemical perturbations on its forecasts and test them against known photochemical relationships. We then examine the internal representations that generate these forecasts. We find that Aurora captures a first-order ozone response to reactive nitrogen but does not enforce the chemical constraints that a process-based model encodes. It generates chemically inconsistent combinations of related species and relaxes localized emission features such as wildfire plumes toward background. Internally, its representations remain largely organized around the meteorology inherited during pretraining, with little structure specific to chemistry. Using sparse autoencoders, we identify internal components that causally control the chemical forecast but do not map cleanly onto individual atmospheric processes. This work provides a framework for testing whether AI forecasting systems learn atmospheric chemistry from reanalysis data. As these models are increasingly positioned to inform environmental policy decisions, we argue that composition forecasts should also be judged by their internal mechanisms rather than by benchmark skill alone.