# A Survey of Deep Learning for Scientific Discovery

arXiv: https://arxiv.org/abs/2003.11755 · HTML: https://mlsanity.com/paper/2003.11755/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2020.

## Built on by (most cited first, 30 of 36)

- [Neural Collapse: A Review on Modelling Principles and Generalization](https://mlsanity.com/paper/2206.04041/) (2022)
- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System](https://mlsanity.com/paper/2410.09403/) (2024)
- [A Review on Explainability in Multimodal Deep Neural Nets](https://mlsanity.com/paper/2105.07878/) (2021)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [Marrying Causal Representation Learning with Dynamical Systems for Science](https://mlsanity.com/paper/2405.13888/) (2024)
- [Controllable Data Generation by Deep Learning: A Review](https://mlsanity.com/paper/2207.09542/) (2022)
- [Multimodal Explainability via Latent Shift applied to COVID-19 stratification](https://mlsanity.com/paper/2212.14084/) (2022)
- [The Future of Fundamental Science Led by Generative Closed-Loop Artificial Intelligence](https://mlsanity.com/paper/2307.07522/) (2023)
- [Tensor Processing Primitives: A Programming Abstraction for Efficiency and Portability in Deep Learning & HPC Workloads](https://mlsanity.com/paper/2104.05755/) (2021)
- [Renormalization in the neural network-quantum field theory correspondence](https://mlsanity.com/paper/2212.11811/) (2022)
- [Advancing the Scientific Method with Large Language Models: From Hypothesis to Discovery](https://mlsanity.com/paper/2505.16477/) (2025)
- [Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentation](https://mlsanity.com/paper/2107.05684/) (2021)
- [When is a System Discoverable from Data? Discovery Requires Chaos](https://mlsanity.com/paper/2511.08860/) (2025)
- [Time Series Analysis and Modeling to Forecast: a Survey](https://mlsanity.com/paper/2104.00164/) (2021)
- [Harnessing Deep Learning and HPC Kernels via High-Level Loop and Tensor Abstractions on CPU Architectures](https://mlsanity.com/paper/2304.12576/) (2023)
- [When less is more: Simplifying inputs aids neural network understanding](https://mlsanity.com/paper/2201.05610/) (2022)
- [Visualizing and Explaining Language Models](https://mlsanity.com/paper/2205.10238/) (2022)
- [Characterizing Coherent Integrated Photonic Neural Networks under Imperfections](https://mlsanity.com/paper/2207.10835/) (2022)
- [A robust estimator of mutual information for deep learning interpretability](https://mlsanity.com/paper/2211.00024/) (2022)
- [Incremental Cross-Domain Adaptation for Robust Retinopathy Screening via Bayesian Deep Learning](https://mlsanity.com/paper/2110.09319/) (2021)
- [LDC: Learning to Generate Research Idea with Dynamic Control](https://mlsanity.com/paper/2412.14626/) (2024)
- [Masked Sinogram Model with Transformer for ill-Posed Computed Tomography Reconstruction: a Preliminary Study](https://mlsanity.com/paper/2209.01356/) (2022)
- [Itsy Bitsy SpiderNet: Fully Connected Residual Network for Fraud Detection](https://mlsanity.com/paper/2105.08120/) (2021)
- [AI Research Associate for Early-Stage Scientific Discovery](https://mlsanity.com/paper/2202.03199/) (2022)
- [Net2Brain: A Toolbox to compare artificial vision models with human brain responses](https://mlsanity.com/paper/2208.09677/) (2022)
- [The Representational Status of Deep Learning Models](https://mlsanity.com/paper/2303.12032/) (2023)
- [Applications of Explainable artificial intelligence in Earth system science](https://mlsanity.com/paper/2406.11882/) (2024)
- [Dynamic Bayesian Optimization Framework for Instruction Tuning in Partial Differential Equation Discovery](https://mlsanity.com/paper/2601.00088/) (2025)
- [Toward Building Science Discovery Machines](https://mlsanity.com/paper/2103.15551/) (2021)

## Abstract (arXiv)

Over the past few years, we have seen fundamental breakthroughs in core problems in machine learning, largely driven by advances in deep neural networks. At the same time, the amount of data collected in a wide array of scientific domains is dramatically increasing in both size and complexity. Taken together, this suggests many exciting opportunities for deep learning applications in scientific settings. But a significant challenge to this is simply knowing where to start. The sheer breadth and diversity of different deep learning techniques makes it difficult to determine what scientific problems might be most amenable to these methods, or which specific combination of methods might offer the most promising first approach. In this survey, we focus on addressing this central issue, providing an overview of many widely used deep learning models, spanning visual, sequential and graph structured data, associated tasks and different training methods, along with techniques to use deep learning with less data and better interpret these complex models --- two central considerations for many scientific use cases. We also include overviews of the full design process, implementation tips, and links to a plethora of tutorials, research summaries and open-sourced deep learning pipelines and pretrained models, developed by the community. We hope that this survey will help accelerate the use of deep learning across different scientific domains.