# Propagating the prior from shallow to deep with a pre-trained velocity-model Generative Transformer network

arXiv: https://arxiv.org/abs/2408.09767 · HTML: https://mlsanity.com/paper/2408.09767/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2024.

## Built on by (most cited first, 2 of 2)

- [Improving Full Waveform Inversion in Large Model Era](https://mlsanity.com/paper/2603.00377/) (2026)
- [Using Seismic Statistical Features and VQ-VAE to Improve Spatiotemporal Seismicity Predictability](https://mlsanity.com/paper/2606.10069/) (2026)

## Abstract (arXiv)

Building subsurface velocity models is essential to our goals in utilizing seismic data for Earth discovery and exploration, as well as monitoring. With the dawn of machine learning, these velocity models (or, more precisely, their distribution) can be stored accurately and efficiently in a generative model. These stored velocity model distributions can be utilized to regularize or quantify uncertainties in inverse problems, like full waveform inversion. However, most generators, like normalizing flows or diffusion models, treat the image (velocity model) uniformly, disregarding spatial dependencies and resolution changes with respect to the observation locations. To address this weakness, we introduce VelocityGPT, a novel implementation that utilizes Transformer decoders trained autoregressively to generate a velocity model from shallow subsurface to deep. Owing to the fact that seismic data are often recorded on the Earth's surface, a top-down generator can utilize the inverted information in the shallow as guidance (prior) to generating the deep. To facilitate the implementation, we use an additional network to compress the velocity model. We also inject prior information, like well or structure (represented by a migration image) to generate the velocity model. Using synthetic data, we demonstrate the effectiveness of VelocityGPT as a promising approach in generative model applications for seismic velocity model building.