# Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Model

arXiv: https://arxiv.org/abs/2312.08403 · HTML: https://mlsanity.com/paper/2312.08403/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 79 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 30 of 44)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction](https://mlsanity.com/paper/1610.00081/) (2016)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Deep multi-scale video prediction beyond mean square error](https://mlsanity.com/paper/1511.05440/) (2015)
- [Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers](https://mlsanity.com/paper/2111.13587/) (2021)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning](https://mlsanity.com/paper/1605.08104/) (2016)
- [Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Prediction](https://mlsanity.com/paper/2003.01460/) (2020)
- [Scaling Autoregressive Video Models](https://mlsanity.com/paper/1906.02634/) (2019)
- [SimVP: Simpler yet Better Video Prediction](https://mlsanity.com/paper/2206.05099/) (2022)
- [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2103.09504/) (2021)
- [Action-Conditional Video Prediction using Deep Networks in Atari Games](https://mlsanity.com/paper/1507.08750/) (2015)
- [Deep Learning for Spatio-Temporal Data Mining: A Survey](https://mlsanity.com/paper/1906.04928/) (2019)
- [Earthformer: Exploring Space-Time Transformers for Earth System Forecasting](https://mlsanity.com/paper/2207.05833/) (2022)
- [Enhancing Computational Fluid Dynamics with Machine Learning](https://mlsanity.com/paper/2110.02085/) (2021)
- [Video (language) modeling: a baseline for generative models of natural videos](https://mlsanity.com/paper/1412.6604/) (2014)
- [Learning to Generate Long-term Future via Hierarchical Prediction](https://mlsanity.com/paper/1704.05831/) (2017)
- [DL-Traff: Survey and Benchmark of Deep Learning Models for Urban Traffic Prediction](https://mlsanity.com/paper/2108.09091/) (2021)
- [Predicting Video with VQVAE](https://mlsanity.com/paper/2103.01950/) (2021)
- [Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles](https://mlsanity.com/paper/2001.11086/) (2020)
- [Future Video Synthesis with Object Motion Prediction](https://mlsanity.com/paper/2004.00542/) (2020)

## Built on by (most cited first, 17 of 17)

- [MolTC: Towards Molecular Relational Modeling In Language Models](https://mlsanity.com/paper/2402.03781/) (2024)
- [EXGC: Bridging Efficiency and Explainability in Graph Condensation](https://mlsanity.com/paper/2402.05962/) (2024)
- [NeuralOM: Neural Ocean Model for Subseasonal-to-Seasonal Simulation](https://mlsanity.com/paper/2505.21020/) (2025)
- [CaT-GNN: Enhancing Credit Card Fraud Detection via Causal Temporal Graph Neural Networks](https://mlsanity.com/paper/2402.14708/) (2024)
- [Spatio-Temporal Fluid Dynamics Modeling via Physical-Awareness and Parameter Diffusion Guidance](https://mlsanity.com/paper/2403.13850/) (2024)
- [BeamVQ: Aligning Space-Time Forecasting Model via Self-training on Physics-aware Metrics](https://mlsanity.com/paper/2405.17051/) (2024)
- [S^2-KD: Semantic-Spectral Knowledge Distillation Spatiotemporal Forecasting](https://mlsanity.com/paper/2512.00366/) (2025)
- [PnP-Corrector: A Universal Correction Framework for Coupled Spatiotemporal Forecasting](https://mlsanity.com/paper/2605.08935/) (2026)
- [Adaptive Runge-Kutta Dynamics for Spatiotemporal Prediction](https://mlsanity.com/paper/2405.14504/) (2024)
- [Learning from History: A Retrieval-Augmented Framework for Spatiotemporal Prediction](https://mlsanity.com/paper/2510.24049/) (2025)
- [Physics-informed Discovery of State Variables in Second-Order and Hamiltonian Systems](https://mlsanity.com/paper/2408.11691/) (2024)
- [VISION: Prompting Ocean Vertical Velocity Reconstruction from Incomplete Observations](https://mlsanity.com/paper/2509.21477/) (2025)
- [Rectifying Distribution Shift in Cascaded Precipitation Nowcasting](https://mlsanity.com/paper/2511.17628/) (2025)
- [REE-TTT: Highly Adaptive Radar Echo Extrapolation Based on Test-Time Training](https://mlsanity.com/paper/2601.01605/) (2026)
- [AsyncCouple-Flow: Asynchronous Cross-Modal Coupling and Flow Matching for Spatio-Temporal Forecasting](https://mlsanity.com/paper/2609.16573/) (2026)
- [Physics-Guided Flow-Map Matching for Precipitation Nowcasting](https://mlsanity.com/paper/2609.37487/) (2026)
- [BridgeCast: Bridging Ocean Wave Forecasts to Reanalysis via Flow Matching with Exogenous Variables](https://mlsanity.com/paper/2610.03759/) (2026)

## Abstract (arXiv)

Efficiently modeling spatio-temporal (ST) physical processes and observations presents a challenging problem for the deep learning community. Many recent studies have concentrated on meticulously reconciling various advantages, leading to designed models that are neither simple nor practical. To address this issue, this paper presents a systematic study on existing shortcomings faced by off-the-shelf models, including lack of local fidelity, poor prediction performance over long time-steps,low scalability, and inefficiency. To systematically address the aforementioned problems, we propose an EarthFarseer, a concise framework that combines parallel local convolutions and global Fourier-based transformer architectures, enabling dynamically capture the local-global spatial interactions and dependencies. EarthFarseer also incorporates a multi-scale fully convolutional and Fourier architectures to efficiently and effectively capture the temporal evolution. Our proposal demonstrates strong adaptability across various tasks and datasets, with fast convergence and better local fidelity in long time-steps predictions. Extensive experiments and visualizations over eight human society physical and natural physical datasets demonstrates the state-of-the-art performance of EarthFarseer. We release our code at https://github.com/easylearningscores/EarthFarseer.