# Deep Learning Opacity in Scientific Discovery

arXiv: https://arxiv.org/abs/2206.00520 · HTML: https://mlsanity.com/paper/2206.00520/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 48 references that are in the map.
- First posted on arXiv 1 Jun 2022.

## Builds on (most cited first, 3 of 3)

- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Understanding artificial intelligence ethics and safety](https://mlsanity.com/paper/1906.05684/) (2019)
- [Machine Learning and the Future of Realism](https://mlsanity.com/paper/1704.04688/) (2017)

## Built on by (most cited first, 9 of 9)

- [Confabulation: The Surprising Value of Large Language Model Hallucinations](https://mlsanity.com/paper/2406.04175/) (2024)
- [Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models](https://mlsanity.com/paper/2501.05382/) (2025)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models](https://mlsanity.com/paper/2510.23191/) (2025)
- [Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking](https://mlsanity.com/paper/2410.08616/) (2024)
- [Reliability and Interpretability in Science and Deep Learning](https://mlsanity.com/paper/2401.07359/) (2024)
- [SIDEs: Separating Idealization from Deceptive Explanations in xAI](https://mlsanity.com/paper/2404.16534/) (2024)
- [From Hallucination to Reliability: Generative Modeling and the Structure of Scientific Inference](https://mlsanity.com/paper/2504.08526/) (2025)
- [The Representational Status of Deep Learning Models](https://mlsanity.com/paper/2303.12032/) (2023)

## Abstract (arXiv)

Philosophers have recently focused on critical, epistemological challenges that arise from the opacity of deep neural networks. One might conclude from this literature that doing good science with opaque models is exceptionally challenging, if not impossible. Yet, this is hard to square with the recent boom in optimism for AI in science alongside a flood of recent scientific breakthroughs driven by AI methods. In this paper, I argue that the disconnect between philosophical pessimism and scientific optimism is driven by a failure to examine how AI is actually used in science. I show that, in order to understand the epistemic justification for AI-powered breakthroughs, philosophers must examine the role played by deep learning as part of a wider process of discovery. The philosophical distinction between the 'context of discovery' and the 'context of justification' is helpful in this regard. I demonstrate the importance of attending to this distinction with two cases drawn from the scientific literature, and show that epistemic opacity need not diminish AI's capacity to lead scientists to significant and justifiable breakthroughs.