# SeisCLIP: A seismology foundation model pre-trained by multi-modal data for multi-purpose seismic feature extraction

arXiv: https://arxiv.org/abs/2309.02320 · HTML: https://mlsanity.com/paper/2309.02320/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 91 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval](https://mlsanity.com/paper/2104.08860/) (2021)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [AudioCLIP: Extending CLIP to Image, Text and Audio](https://mlsanity.com/paper/2106.13043/) (2021)
- [SSAST: Self-Supervised Audio Spectrogram Transformer](https://mlsanity.com/paper/2110.09784/) (2021)
- [Contrastive Audio-Visual Masked Autoencoder](https://mlsanity.com/paper/2210.07839/) (2022)
- [Audio Transformers](https://mlsanity.com/paper/2105.00335/) (2021)
- [Seismic Signal Denoising and Decomposition Using Deep Neural Networks](https://mlsanity.com/paper/1811.02695/) (2018)
- [Bayesian-Deep-Learning Estimation of Earthquake Location from Single-Station Observations](https://mlsanity.com/paper/1912.01144/) (2019)
- [CRED: A Deep Residual Network of Convolutional and Recurrent Units for Earthquake Signal Detection](https://mlsanity.com/paper/1810.01965/) (2018)
- [PhaseLink: A Deep Learning Approach to Seismic Phase Association](https://mlsanity.com/paper/1809.02880/) (2018)
- [Earthquake Phase Association with Graph Neural Networks](https://mlsanity.com/paper/2209.07086/) (2022)
- [Deep learning for seismic phase detection and picking in the aftershock zone of 2008 Mw7.9 Wenchuan earthquake](https://mlsanity.com/paper/1901.06396/) (2019)
- [StorSeismic: A new paradigm in deep learning for seismic processing](https://mlsanity.com/paper/2205.00222/) (2022)
- [Optimizing a Transformer-based network for a deep learning seismic processing workflow](https://mlsanity.com/paper/2308.04739/) (2023)
- [Combining Deep Learning with Physics Based Features in Explosion-Earthquake Discrimination](https://mlsanity.com/paper/2203.06347/) (2022)
- [Multi-task multi-station earthquake monitoring: An all-in-one seismic Phase picking, Location, and Association Network (PLAN)](https://mlsanity.com/paper/2306.13918/) (2023)

## Built on by (most cited first, 8 of 8)

- [Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review](https://mlsanity.com/paper/2506.01364/) (2025)
- [SeisLM: a Foundation Model for Seismic Waveforms](https://mlsanity.com/paper/2410.15765/) (2024)
- [SeisT: A foundational deep learning model for earthquake monitoring tasks](https://mlsanity.com/paper/2310.01037/) (2023)
- [Evaluation of Seismic Artificial Intelligence with Uncertainty](https://mlsanity.com/paper/2501.14809/) (2025)
- [SeisMoLLM: Advancing Seismic Monitoring via Cross-modal Transfer with Pre-trained Large Language Model](https://mlsanity.com/paper/2502.19960/) (2025)
- [Augmenting representations with scientific papers](https://mlsanity.com/paper/2603.04516/) (2026)
- [Improving moment tensor solutions under Earth structure uncertainty with simulation-based inference](https://mlsanity.com/paper/2603.18925/) (2026)
- [Earth Science Foundation Models: From Perception to Reasoning and Discovery](https://mlsanity.com/paper/2605.12542/) (2026)

## Abstract (arXiv)

Training specific deep learning models for particular tasks is common across various domains within seismology. However, this approach encounters two limitations: inadequate labeled data for certain tasks and limited generalization across regions. To address these challenges, we develop SeisCLIP, a seismology foundation model trained through contrastive learning from multi-modal data. It consists of a transformer encoder for extracting crucial features from time-frequency seismic spectrum and an MLP encoder for integrating the phase and source information of the same event. These encoders are jointly pre-trained on a vast dataset and the spectrum encoder is subsequently fine-tuned on smaller datasets for various downstream tasks. Notably, SeisCLIP's performance surpasses that of baseline methods in event classification, localization, and focal mechanism analysis tasks, employing distinct datasets from different regions. In conclusion, SeisCLIP holds significant potential as a foundational model in the field of seismology, paving the way for innovative directions in foundation-model-based seismology research.