# Improving Full Waveform Inversion in Large Model Era

arXiv: https://arxiv.org/abs/2603.00377 · HTML: https://mlsanity.com/paper/2603.00377/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 47 references that are in the map.
- First posted on arXiv 27 Feb 2026.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [MMaDA: Multimodal Large Diffusion Language Models](https://mlsanity.com/paper/2505.15809/) (2025)
- [Sequential Modeling Enables Scalable Learning for Large Vision Models](https://mlsanity.com/paper/2312.00785/) (2023)
- [Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing](https://mlsanity.com/paper/2407.01521/) (2024)
- [InversionNet: A Real-Time and Accurate Full Waveform Inversion with CNNs and continuous CRFs](https://mlsanity.com/paper/1811.07875/) (2018)
- [OpenFWI: Large-Scale Multi-Structural Benchmark Datasets for Seismic Full Waveform Inversion](https://mlsanity.com/paper/2111.02926/) (2021)
- [Deep-Learning Inversion of Seismic Data](https://mlsanity.com/paper/1901.07733/) (2019)
- [A prior regularized full waveform inversion using generative diffusion models](https://mlsanity.com/paper/2306.12776/) (2023)
- [Unsupervised Learning of Full-Waveform Inversion: Connecting CNN and Partial Differential Equation in a Loop](https://mlsanity.com/paper/2110.07584/) (2021)
- [An Empirical Study of Large-Scale Data-Driven Full Waveform Inversion](https://mlsanity.com/paper/2307.15388/) (2023)
- [An Intriguing Property of Geophysics Inversion](https://mlsanity.com/paper/2204.13731/) (2022)
- [Learned Full Waveform Inversion Incorporating Task Information for Ultrasound Computed Tomography](https://mlsanity.com/paper/2308.16290/) (2023)
- [Auto-Linear Phenomenon in Subsurface Imaging](https://mlsanity.com/paper/2305.13314/) (2023)
- [Propagating the prior from shallow to deep with a pre-trained velocity-model Generative Transformer network](https://mlsanity.com/paper/2408.09767/) (2024)

## Abstract (arXiv)

Full Waveform Inversion (FWI) is a highly nonlinear and ill-posed problem that aims to recover subsurface velocity maps from surface-recorded seismic waveforms data. Existing data-driven FWI typically uses small models, as available datasets have limited volume, geological diversity, and spatial extent, leading to substantial concerns about overfitting. Although they perform well on synthetic datasets, current methods fail to generalize to more realistic geological structures. In this work, we show that a model trained entirely on simulated and relatively simple data can generalize remarkably well to challenging and unseen geological benchmarks. We provide a working recipe that tames a billion-parameter model for FWI through coordinated scaling across three axes: model capacity, data diversity, and training strategy. Our model achieves state-of-the-art performance on OpenFWI and significantly narrows the generalization gap in data-driven FWI. Across six challenging geophysical benchmarks, including Marmousi, 2D SEG/EAGE Salt and Overthrust, 2004 BP, Sigsbee, and SEAM Phase I, it infers complex structures absent from the training set and delivers significant performance improvements (SSIM from 0.5844 to 0.7669). Overall, our results demonstrate that with an appropriate scaling strategy, large models trained on simple synthetic data can achieve substantial generalization to more complex and realistic geological structures.