# Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design

arXiv: https://arxiv.org/abs/2303.02884 · HTML: https://mlsanity.com/paper/2303.02884/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 81 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts](https://mlsanity.com/paper/2110.01691/) (2021)
- [Jury Learning: Integrating Dissenting Voices into Machine Learning Models](https://mlsanity.com/paper/2202.02950/) (2022)
- [Problem Formulation and Fairness](https://mlsanity.com/paper/1901.02547/) (2019)
- [Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI](https://mlsanity.com/paper/1909.02309/) (2019)
- [Designing Toxic Content Classification for a Diversity of Perspectives](https://mlsanity.com/paper/2106.04511/) (2021)
- [How do Data Science Workers Collaborate? Roles, Workflows, and Tools](https://mlsanity.com/paper/2001.06684/) (2020)
- [Machine Teaching: A New Paradigm for Building Machine Learning Systems](https://mlsanity.com/paper/1707.06742/) (2017)
- [Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution](https://mlsanity.com/paper/2012.08290/) (2020)
- [How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?](https://mlsanity.com/paper/1909.03486/) (2019)
- [How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study](https://mlsanity.com/paper/2101.06098/) (2021)
- [Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects](https://mlsanity.com/paper/2002.03389/) (2020)
- [Data Vision: Learning to See Through Algorithmic Abstraction](https://mlsanity.com/paper/2002.03387/) (2020)
- [Hypothesis Formalization: Empirical Findings, Software Limitations, and Design Implications](https://mlsanity.com/paper/2104.02712/) (2021)
- [User Modelling for Avoiding Overfitting in Interactive Knowledge Elicitation for Prediction](https://mlsanity.com/paper/1710.04881/) (2017)

## Built on by (most cited first, 15 of 15)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testing](https://mlsanity.com/paper/2309.09128/) (2023)
- [Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM](https://mlsanity.com/paper/2404.12259/) (2024)
- [Designing LLM Chains by Adapting Techniques from Crowdsourcing Workflows](https://mlsanity.com/paper/2312.11681/) (2023)
- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [Beyond Testers' Biases: Guiding Model Testing with Knowledge Bases using LLMs](https://mlsanity.com/paper/2310.09668/) (2023)
- [Clarify: Improving Model Robustness With Natural Language Corrections](https://mlsanity.com/paper/2402.03715/) (2024)
- [Unpacking Human-AI interactions: From interaction primitives to a design space](https://mlsanity.com/paper/2401.05115/) (2024)
- [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025)
- [Policy Maps: Tools for Guiding the Unbounded Space of LLM Behaviors](https://mlsanity.com/paper/2409.18203/) (2024)
- [SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks](https://mlsanity.com/paper/2305.03039/) (2023)
- [Seizing the Means of Production: Exploring the Landscape of Crafting, Adapting and Navigating Generative AI Models in the Visual Arts](https://mlsanity.com/paper/2404.17688/) (2024)
- [Scaffolding Recursive Divergence and Convergence in Story Ideation](https://mlsanity.com/paper/2507.03307/) (2025)
- [AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages](https://mlsanity.com/paper/2505.10300/) (2025)
- [Exploring the Innovation Opportunities for Pre-trained Models](https://mlsanity.com/paper/2505.15790/) (2025)

## Abstract (arXiv)

Machine learning practitioners often end up tunneling on low-level technical details like model architectures and performance metrics. Could early model development instead focus on high-level questions of which factors a model ought to pay attention to? Inspired by the practice of sketching in design, which distills ideas to their minimal representation, we introduce model sketching: a technical framework for iteratively and rapidly authoring functional approximations of a machine learning model's decision-making logic. Model sketching refocuses practitioner attention on composing high-level, human-understandable concepts that the model is expected to reason over (e.g., profanity, racism, or sarcasm in a content moderation task) using zero-shot concept instantiation. In an evaluation with 17 ML practitioners, model sketching reframed thinking from implementation to higher-level exploration, prompted iteration on a broader range of model designs, and helped identify gaps in the problem formulationunicodex2014all in a fraction of the time ordinarily required to build a model.