# Data-Driven Forecasting of three-Component Seismograms Using Transformer Architectures

arXiv: https://arxiv.org/abs/2606.02912 · HTML: https://mlsanity.com/paper/2606.02912/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 66 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysis](https://mlsanity.com/paper/2402.16412/) (2024)
- [Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization](https://mlsanity.com/paper/2412.05244/) (2024)
- [Seismic Signal Denoising and Decomposition Using Deep Neural Networks](https://mlsanity.com/paper/1811.02695/) (2018)
- [SeismicNet: Physics-informed neural networks for seismic wave modeling in semi-infinite domain](https://mlsanity.com/paper/2210.14044/) (2022)
- [SeisLM: a Foundation Model for Seismic Waveforms](https://mlsanity.com/paper/2410.15765/) (2024)
- [TOKON: TOKenization-Optimized Normalization for time series analysis with a large language model](https://mlsanity.com/paper/2502.05701/) (2025)
- [Multiple-Resolution Tokenization for Time Series Forecasting with an Application to Pricing](https://mlsanity.com/paper/2407.03185/) (2024)
- [Forecasting Seismic Waveforms: A Deep Learning Approach for Einstein Telescope](https://mlsanity.com/paper/2509.21446/) (2025)

## Built on by (most cited first, 1 of 1)

- [When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismograms](https://mlsanity.com/paper/2606.10868/) (2026)

## Abstract (arXiv)

Forecasting seismic waveforms beyond observed data remains challenging due to the nonlinear, dispersive, and multi-scale nature of seismic wave propagation. In this work, we introduce textscSeismoGPT, a transformer-based autoregressive model designed to forecast three-component seismic waveforms directly in the time domain. Forecasting is formulated as a physically constrained continuation problem in which the model receives waveform context beginning at the P-wave arrival and extending a defined time beyond the S-wave arrival, after which future motion is generated recursively without access to ground-truth samples. Evaluation is performed on synthetic seismograms spanning source depths of 5--100,km, epicentral distances of 10--90^circ, and magnitudes 3 ≤ Mw ≤ 7. To disentangle the effects of context length and prediction horizon, we define three evaluation configurations using a distance-normalized context ratio and fixed prediction horizons of 120 and 240,s. Across all configurations, the model achieves a median normalized cross correlation of 0.93 or higher. Analysis of representative forecasts shows that successful predictions preserve both phase coherence and spectral energy distribution. Where failure cases arise, this is primarily due to gradual phase drift during autoregressive rollout rather than unphysical signal generation. These results demonstrate that transformer-based sequence models can learn stable dynamical continuation of seismic wavefields, highlighting the potential of foundation-model approaches for physics-driven time-series forecasting. There are potential applications of this methodology in seismic warning and hazard mitigation, particularly for next-generation gravitational-wave observatories, such as the Einstein Telescope.