mlsanityData to 14 Oct 2026

In-context learning enables continental-scale subsurface temperature prediction from sparse local observations

arXiv 2605.16665 ↗First posted 15 May 2026cs.LG physics.geo-phcited by 0

Not on the map. Why?

Abstract (arXiv)

Continental-scale knowledge of subsurface temperature is limited by the cost and sparsity of borehole measurements, but such information is essential for geothermal resource assessment and for understanding heat transport in the shallow crust. The thermal field reflects the interaction between lithology, crustal structure, radiogenic heat production, and advective fluid flow, sometimes producing sharp anomalies that are smoothed by conventional interpolation or difficult to capture with physical models. Here we introduce In-Context Earth, a transformer-based model that uses sparse local borehole observations as geological context to predict continuous temperature-at-depth fields with calibrated uncertainty. In the contiguous United States, the model achieves a mean absolute error of 4.7 degC, outperforming the physics-informed Stanford Thermal Model, a model based on AlphaEarth embeddings, the multimodal Transparent Earth model, and universal kriging, while resolving sharper thermal gradients in geothermal provinces. Its uncertainty estimates are well calibrated, with a Kolmogorov-Smirnov statistic of 2.5%. Without finetuning, the model adapts to Alberta, Australia, and the United Kingdom (UK) using only 20 local observations at inference time, maintaining high accuracy in geologically distinct test regions with a mean absolute error of 2.2 degC in Alberta, 6.2 degC in Australia, and 5.4 degC in the UK. Interpretability analyses show that the model learns internal representations of subsurface properties it never observes during training, including seismic velocities, geochemistry, and crustal structure, and uses these representations in physically consistent ways. More broadly, this work shows that in-context learning can use sparse borehole observations for continental-scale subsurface characterization, without requiring dense measurements or region-specific retraining.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 36 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

10 of 36 references.

References · 10

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  6. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  7. 2025AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label datacited by 137
  8. 2025The Bayesian Geometry of Transformer Attentioncited by 13
  9. 2024Thermal Earth Model for the Conterminous United States Using an Interpolative Physics-Informed Graph Neural Network (InterPIGNN)cited by 2
  10. 2025The Transparent Earth: A Multimodal Foundation Model for the Earth's Subsurfacecited by 1

Built on by

Not cited yet.