Scalable Artificial Intelligence for Science: Perspectives, Methods and Exemplars
Not on the map. Why?
Abstract (arXiv)
In a post-ChatGPT world, this paper explores the potential of leveraging scalable artificial intelligence for scientific discovery. We propose that scaling up artificial intelligence on high-performance computing platforms is essential to address such complex problems. This perspective focuses on scientific use cases like cognitive simulations, large language models for scientific inquiry, medical image analysis, and physics-informed approaches. The study outlines the methodologies needed to address such challenges at scale on supercomputers or the cloud and provides exemplars of such approaches applied to solve a variety of scientific problems.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 48 of its 108 references that are in the map. First posted on arXiv 24 Jun 2024.
Builds on
48 of 108 references.
References · 48
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
- 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
- 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
- 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
36 more
- 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
- 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
- 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
- 2021Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LMcited by 621
- 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
- 2022Efficiently Scaling Transformer Inferencecited by 529
- 2020Time Series Forecasting With Deep Learning: A Surveycited by 457
- 2019Augmented Neural ODEscited by 396
- 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
- 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
- 2020Universal Differential Equations for Scientific Machine Learningcited by 345
- 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 335
- 2018Horovod: fast and easy distributed deep learning in TensorFlowcited by 275
- 2021Neural Symbolic Regression that Scalescited by 185
- 2022Uncertainty Quantification in Scientific Machine Learning: Methods, Metrics, and Comparisonscited by 169
- 2018Mesh-TensorFlow: Deep Learning for Supercomputerscited by 168
- 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
- 2018Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysiscited by 154
- 2020Dissecting Neural ODEscited by 150
- 2018Adversarial Uncertainty Quantification in Physics-Informed Neural Networkscited by 120
- 2020Multiscale Simulations of Complex Systems by Learning their Effective Dynamicscited by 86
- 2017TensorFlow-Serving: Flexible, High-Performance ML Servingcited by 85
- 2021Scientific Machine Learning Benchmarkscited by 60
- 2018Exascale Deep Learning for Climate Analyticscited by 28
- 2017Integrated Model, Batch and Domain Parallelism in Training Neural Networkscited by 16
- 2023Optimizing Distributed Training on Frontier for Large Language Modelscited by 12
- 2020The Case for Strong Scaling in Deep Learning: Training Large 3D CNNs with Hybrid Parallelismcited by 11
- 2021Colmena: Scalable Machine-Learning-Based Steering of Ensemble Simulations for High Performance Computingcited by 10
- 2019High Resolution Medical Image Analysis with Spatial Partitioningcited by 9
- 2019Improving Strong-Scaling of CNN Training by Exploiting Finer-Grained Parallelismcited by 8
- 2024AI-coupled HPC Workflow Applications, Middleware and Performancecited by 7
- 2022Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature reviewcited by 6
- 2021PI3NN: Out-of-distribution-aware prediction intervals from three neural networkscited by 5
- 2021AI and extreme scale computing to learn and infer the physics of higher order gravitational wave modes of quasi-circular, spinning, non-precessing binary black hole mergerscited by 3
- 2021Is Disaggregation possible for HPC Cognitive Simulation?cited by 1
Built on by
Citing papers per year: 2025: 2.
2 papers.