# Symphony: Composing Interactive Interfaces for Machine Learning

arXiv: https://arxiv.org/abs/2202.08946 · HTML: https://mlsanity.com/paper/2202.08946/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 63 references that are in the map.
- First posted on arXiv 18 Feb 2022.

## Builds on (most cited first, 19 of 19)

- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [Machine Learning Testing: Survey, Landscapes and Horizons](https://mlsanity.com/paper/1906.10742/) (2019)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://mlsanity.com/paper/1801.06889/) (2018)
- [The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards](https://mlsanity.com/paper/1805.03677/) (2018)
- [FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning](https://mlsanity.com/paper/1904.05419/) (2019)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations](https://mlsanity.com/paper/1904.02323/) (2019)
- [Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models](https://mlsanity.com/paper/1804.09299/) (2018)
- [FairSight: Visual Analytics for Fairness in Decision Making](https://mlsanity.com/paper/1908.00176/) (2019)
- [ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models](https://mlsanity.com/paper/1704.01942/) (2017)
- [OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples](https://mlsanity.com/paper/2002.03103/) (2020)
- [ConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion](https://mlsanity.com/paper/1910.00969/) (2019)
- [mage: Fluid Moves Between Code and Graphical Work in Computational Notebooks](https://mlsanity.com/paper/2009.10643/) (2020)
- [Ablate, Variate, and Contemplate: Visual Analytics for Discovering Neural Architectures](https://mlsanity.com/paper/1908.00387/) (2019)

## Built on by (most cited first, 18 of 18)

- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitioners](https://mlsanity.com/paper/2304.00078/) (2023)
- [Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences](https://mlsanity.com/paper/2310.04621/) (2023)
- [Detecting Label Errors by using Pre-Trained Language Models](https://mlsanity.com/paper/2205.12702/) (2022)
- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks](https://mlsanity.com/paper/2302.11703/) (2023)
- [Abstraction Alignment: Comparing Model-Learned and Human-Encoded Conceptual Relationships](https://mlsanity.com/paper/2407.12543/) (2024)
- [Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference](https://mlsanity.com/paper/2404.03085/) (2024)
- [Allowing humans to interactively guide machines where to look does not always improve human-AI team's classification accuracy](https://mlsanity.com/paper/2404.05238/) (2024)
- [Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts](https://mlsanity.com/paper/2205.05749/) (2022)
- [Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments](https://mlsanity.com/paper/2408.03274/) (2024)
- [SuperNOVA: Design Strategies and Opportunities for Interactive Visualization in Computational Notebooks](https://mlsanity.com/paper/2305.03039/) (2023)
- [RAG Without the Lag: Interactive Debugging for Retrieval-Augmented Generation Pipelines](https://mlsanity.com/paper/2504.13587/) (2025)
- [Interactivity x Explainability: Toward Understanding How Interactivity Can Improve Computer Vision Explanations](https://mlsanity.com/paper/2504.10745/) (2025)
- [Towards Multifaceted Human-Centered AI](https://mlsanity.com/paper/2301.03656/) (2023)
- [What Do People Want to Know About Artificial Intelligence (AI)? The Importance of Answering End-User Questions to Explain Autonomous Vehicle (AV) Decisions](https://mlsanity.com/paper/2505.06428/) (2025)
- [FAIRTOPIA: Envisioning Multi-Agent Guardianship for Disrupting Unfair AI Pipelines](https://mlsanity.com/paper/2506.09107/) (2025)
- [AI Supply Chain Galaxy: 3D Visual Analytics for License Compliance](https://mlsanity.com/paper/2606.16292/) (2026)

## Abstract (arXiv)

Interfaces for machine learning (ML), information and visualizations about models or data, can help practitioners build robust and responsible ML systems. Despite their benefits, recent studies of ML teams and our interviews with practitioners (n=9) showed that ML interfaces have limited adoption in practice. While existing ML interfaces are effective for specific tasks, they are not designed to be reused, explored, and shared by multiple stakeholders in cross-functional teams. To enable analysis and communication between different ML practitioners, we designed and implemented Symphony, a framework for composing interactive ML interfaces with task-specific, data-driven components that can be used across platforms such as computational notebooks and web dashboards. We developed Symphony through participatory design sessions with 10 teams (n=31), and discuss our findings from deploying Symphony to 3 production ML projects at Apple. Symphony helped ML practitioners discover previously unknown issues like data duplicates and blind spots in models while enabling them to share insights with other stakeholders.