# Intelligent Sampling of Extreme-Scale Turbulence Datasets for Accurate and Efficient Spatiotemporal Model Training

arXiv: https://arxiv.org/abs/2508.03872 · HTML: https://mlsanity.com/paper/2508.03872/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 75 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks](https://mlsanity.com/paper/2007.04542/) (2020)
- [The rising costs of training frontier AI models](https://mlsanity.com/paper/2405.21015/) (2024)
- [A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility](https://mlsanity.com/paper/2504.07086/) (2025)
- [Combiner: Full Attention Transformer with Sparse Computation Cost](https://mlsanity.com/paper/2107.05768/) (2021)
- [Data-Driven Sparse Sensor Placement for Reconstruction](https://mlsanity.com/paper/1701.07569/) (2017)
- [Entropy Law: The Story Behind Data Compression and LLM Performance](https://mlsanity.com/paper/2407.06645/) (2024)
- [FedGraph: Federated Graph Learning with Intelligent Sampling](https://mlsanity.com/paper/2111.01370/) (2021)
- [A Perspective on Machine Learning Methods in Turbulence Modelling](https://mlsanity.com/paper/2010.12226/) (2020)
- [Evaluation of Sampling Methods for Scatterplots](https://mlsanity.com/paper/2007.14666/) (2020)
- [AI-coupled HPC Workflow Applications, Middleware and Performance](https://mlsanity.com/paper/2406.14315/) (2024)
- [Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows](https://mlsanity.com/paper/2112.15446/) (2021)
- [Polytope: An Algorithm for Efficient Feature Extraction on Hypercubes](https://mlsanity.com/paper/2306.11553/) (2023)

## Abstract (arXiv)

With the end of Moore's law and Dennard scaling, efficient training increasingly requires rethinking data volume. Can we train better models with significantly less data via intelligent subsampling? To explore this, we develop SICKLE, a sparse intelligent curation framework for efficient learning, featuring a novel maximum entropy (MaxEnt) sampling approach, scalable training, and energy benchmarking. We compare MaxEnt with random and phase-space sampling on large direct numerical simulation (DNS) datasets of turbulence. Evaluating SICKLE at scale on Frontier, we show that subsampling as a preprocessing step can, in many cases, improve model accuracy and substantially lower energy consumption, with observed reductions of up to 38x.