Deep Learning Opacity in Scientific Discovery
Not on the map. Why?
Abstract (arXiv)
Philosophers have recently focused on critical, epistemological challenges that arise from the opacity of deep neural networks. One might conclude from this literature that doing good science with opaque models is exceptionally challenging, if not impossible. Yet, this is hard to square with the recent boom in optimism for AI in science alongside a flood of recent scientific breakthroughs driven by AI methods. In this paper, I argue that the disconnect between philosophical pessimism and scientific optimism is driven by a failure to examine how AI is actually used in science. I show that, in order to understand the epistemic justification for AI-powered breakthroughs, philosophers must examine the role played by deep learning as part of a wider process of discovery. The philosophical distinction between the 'context of discovery' and the 'context of justification' is helpful in this regard. I demonstrate the importance of attending to this distinction with two cases drawn from the scientific literature, and show that epistemic opacity need not diminish AI's capacity to lead scientists to significant and justifiable breakthroughs.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 3 of its 48 references that are in the map. First posted on arXiv 1 Jun 2022.
Builds on
3 of 48 references.
References · 3
- 2016The Mythos of Model Interpretabilitycited by 716
- 2019Understanding artificial intelligence ethics and safetycited by 68
- 2017Machine Learning and the Future of Realismcited by 4
Built on by
Citing papers per year: 2023: 1; 2024: 4; 2025: 4.
9 papers.
- 2024Confabulation: The Surprising Value of Large Language Model Hallucinationscited by 42
- 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Modelscited by 14
- 2025The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Modelscited by 12
- 2025Interpretable Machine Learning in Physics: A Reviewcited by 11
- 2024Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Brakingcited by 5
- 2024Reliability and Interpretability in Science and Deep Learningcited by 2
- 2024SIDEs: Separating Idealization from Deceptive Explanations in xAIcited by 2
- 2025From Hallucination to Reliability: Generative Modeling and the Structure of Scientific Inferencecited by 2
- 2023The Representational Status of Deep Learning Modelscited by 1
- 2025The Benchmarking Epistemology: Validity Theory for Evaluating Machine Learning Models27 Oct 2025
- 2025From Hallucination to Reliability: Generative Modeling and the Structure of Scientific Inference11 Apr 2025
- 2025Interpretable Machine Learning in Physics: A Review30 Mar 2025
- 2025Large Physics Models: Towards a collaborative approach with Large Language Models and Foundation Models9 Jan 2025
- 2024Dual-AEB: Synergizing Rule-Based and Multimodal Large Language Models for Effective Emergency Braking11 Oct 2024
- 2024Confabulation: The Surprising Value of Large Language Model Hallucinations6 Jun 2024
- 2024SIDEs: Separating Idealization from Deceptive Explanations in xAI25 Apr 2024
- 2024Reliability and Interpretability in Science and Deep Learning14 Jan 2024
- 2023The Representational Status of Deep Learning Models21 Mar 2023