mlsanityData to 7 Oct 2026

Explain the Black Box for the Sake of Science: the Scientific Method in the Era of Generative Artificial Intelligence

arXiv 2406.10557 ↗First posted 15 Jun 2024cs.AI cs.CY math.DScited by 5

Not on the map. Why?

Abstract (arXiv)

The scientific method is the cornerstone of human progress across all branches of the natural and applied sciences, from understanding the human body to explaining how the universe works. The scientific method is based on identifying systematic rules or principles that describe the phenomenon of interest in a reproducible way that can be validated through experimental evidence. In the era of generative artificial intelligence, there are discussions on how AI systems may discover new knowledge. We argue that human complex reasoning for scientific discovery remains of vital importance, at least before the advent of artificial general intelligence. Yet, AI can be leveraged for scientific discovery via explainable AI. More specifically, knowing the `principles' the AI systems used to make decisions can be a point of contact with domain experts and scientists, that can lead to divergent or convergent views on a given scientific problem. Divergent views may spark further scientific investigations leading to interpretability-guided explanations (IGEs), and possibly to new scientific knowledge. We define this field as Explainable AI for Science, where domain experts -- potentially assisted by generative AI -- formulate scientific hypotheses and explanations based on the interpretability of a predictive AI system.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 20 of its 66 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

20 of 66 references.

References · 20

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2018Sanity Checks for Saliency Mapscited by 1,149
  3. 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
  4. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  5. 2020Neural Additive Models: Interpretable Machine Learning with Neural Netscited by 318
  6. 2019Explanations can be manipulated and geometry is to blamecited by 215
  7. 2015Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction modelcited by 140
  8. 2024How Transformers Learn Causal Structure with Gradient Descentcited by 114
  9. 2023Large Language Models for Automated Open-domain Scientific Hypotheses Discoverycited by 100
  10. 2019Unifying machine learning and quantum chemistry -- a deep neural network for molecular wavefunctionscited by 73
  11. 2022On scientific understanding with artificial intelligencecited by 68
  12. 2022Evaluation of post-hoc interpretability methods in time-series classificationcited by 53
8 more
  1. 2021Machine Learning in the Search for New Fundamental Physicscited by 50
  2. 2015Interpretable Classification Models for Recidivism Predictioncited by 44
  3. 2023Causal Interpretation of Self-Attention in Pre-Trained Transformerscited by 30
  4. 2020Causality Learning: A New Perspective for Interpretable Machine Learningcited by 28
  5. 2023Superhuman Artificial Intelligence Can Improve Human Decision Making by Increasing Noveltycited by 27
  6. 2022Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomenacited by 21
  7. 2023The role of causality in explainable artificial intelligencecited by 17
  8. 2024ProtoS-ViT: Visual foundation models for sparse self-explainable classificationscited by 8

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024Explainable Natural Language Processing for Corporate Sustainability Analysiscited by 9
  2. 2024Revisiting the robustness of post-hoc interpretability methodscited by 5
  3. 2025XAI4Extremes: An interpretable machine learning framework for understanding extreme-weather precursors under climate changecited by 2
  4. 2025Explainability matters: The effect of liability rules on the healthcare sectorcited by 1
  5. 2025How are Scientific Concepts Birthed? Typing Rules of Concept Formation in Theoretical Physics Reasoningcited by 0