Model Science: getting serious about verification, explanation and control of AI systems
Not on the map. Why?
Abstract (arXiv)
The growing adoption of foundation models calls for a paradigm shift from Data Science to Model Science. Unlike data-centric approaches, Model Science places the trained model at the core of analysis, aiming to interact, verify, explain, and control its behavior across diverse operational contexts. This paper introduces a conceptual framework for a new discipline called Model Science, along with the proposal for its four key pillars: Verification, which requires strict, context-aware evaluation protocols; Explanation, which is understood as various approaches to explore of internal model operations; Control, which integrates alignment techniques to steer model behavior; and Interface, which develops interactive and visual explanation tools to improve human calibration and decision-making. The proposed framework aims to guide the development of credible, safe, and human-aligned AI systems.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 50 of its 66 references that are in the map. First posted on arXiv 27 Aug 2025.
Builds on
50 of 66 references.
References · 50
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
- 2022Mass-Editing Memory in a Transformercited by 1,001
- 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
- 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
38 more
- 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
- 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
- 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 196
- 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 150
- 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
- 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
- 2024Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)cited by 119
- 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 103
- 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
- 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
- 2025Interpreting CLIP with Hierarchical Sparse Autoencoderscited by 60
- 2024Careless Whisper: Speech-to-Text Hallucination Harmscited by 53
- 2023AttentionViz: A Global View of Transformer Attentioncited by 50
- 2020Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Modelscited by 49
- 2023Adversarial attacks and defenses in explainable artificial intelligence: A surveycited by 48
- 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 46
- 2018Visual Analytics for Explainable Deep Learningcited by 39
- 2023How Does Attention Work in Vision Transformers? A Visual Analytics Attemptcited by 30
- 2020Transparency, Auditability and eXplainability of Machine Learning Models in Credit Scoringcited by 26
- 2023Assessing the Security of GitHub Copilot Generated Code -- A Targeted Replication Studycited by 26
- 2020The Grammar of Interactive Explanatory Model Analysiscited by 24
- 2024The Clever Hans Effect in Unsupervised Learningcited by 24
- 2022Give Me Your Attention: Dot-Product Attention Considered Harmful for Adversarial Patch Robustnesscited by 23
- 2024Position: Explain to Question not to Justifycited by 20
- 2025Causal Head Gating: A Framework for Interpreting Roles of Attention Heads in Transformerscited by 16
- 2020Checklist for responsible deep learning modeling of medical images based on COVID-19 detection studiescited by 15
- 2024Global Counterfactual Directionscited by 15
- 2024Rethinking Visual Counterfactual Explanations Through Region Constraintcited by 14
- 2020What Would You Ask the Machine Learning Model? Identification of User Needs for Model Explanations Based on Human-Model Conversationscited by 11
- 2024The Dark Patterns of Personalized Persuasion in Large Language Models: Exposing Persuasive Linguistic Features for Big Five Personality Traits in LLMs Responsescited by 9
- 2022Protein language models trained on multiple sequence alignments learn phylogenetic relationshipscited by 7
- 2025From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliancecited by 6
- 2025MASCOTS: Model-Agnostic Symbolic COunterfactual explanations for Time Seriescited by 5
- 2025Adultification Bias in LLMs and Text-to-Image Modelscited by 5
- 2023ExplainableFold: Understanding AlphaFold Prediction with Explainable AIcited by 4
- 2025Birds look like cars: Adversarial analysis of intrinsically interpretable deep learningcited by 3
- 2021LIMEcraft: Handcrafted superpixel selection and inspection for Visual eXplanationscited by 2
- 2022Exploring Local Explanations of Nonlinear Models Using Animated Linear Projectionscited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.