# Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience

arXiv: https://arxiv.org/abs/2302.10395 · HTML: https://mlsanity.com/paper/2302.10395/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 106 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 30 of 40)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [Explainable Machine Learning in Deployment](https://mlsanity.com/paper/1909.06342/) (2019)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [Participation is not a Design Fix for Machine Learning](https://mlsanity.com/paper/2007.02423/) (2020)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs](https://mlsanity.com/paper/2004.11440/) (2020)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [Automated Data Slicing for Model Validation:A Big data - AI Integration Approach](https://mlsanity.com/paper/1807.06068/) (2018)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [Interactive Model Cards: A Human-Centered Approach to Model Documentation](https://mlsanity.com/paper/2205.02894/) (2022)
- [Text Summarization Techniques: A Brief Survey](https://mlsanity.com/paper/1707.02268/) (2017)
- [A Survey on Dialogue Summarization: Recent Advances and New Frontiers](https://mlsanity.com/paper/2107.03175/) (2021)
- [Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs](https://mlsanity.com/paper/2103.06076/) (2021)
- [Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AI](https://mlsanity.com/paper/2206.10847/) (2022)
- [Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices](https://mlsanity.com/paper/2006.12358/) (2020)
- [The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems](https://mlsanity.com/paper/2105.03354/) (2021)

## Built on by (most cited first, 18 of 18)

- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [Rethinking Model Evaluation as Narrowing the Socio-Technical Gap](https://mlsanity.com/paper/2306.03100/) (2023)
- [Understanding the LLM-ification of CHI: Unpacking the Impact of LLMs at CHI through a Systematic Literature Review](https://mlsanity.com/paper/2501.12557/) (2025)
- [Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice](https://mlsanity.com/paper/2306.06542/) (2023)
- [Taxonomy of User Needs and Actions](https://mlsanity.com/paper/2510.06124/) (2025)
- [VRCopilot: Authoring 3D Layouts with Generative AI Models in VR](https://mlsanity.com/paper/2408.09382/) (2024)
- [PromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers' Workflows](https://mlsanity.com/paper/2310.15435/) (2023)
- [Attention to Non-Adopters](https://mlsanity.com/paper/2510.15951/) (2025)
- [Prototyping Multimodal GenAI Real-Time Agents with Counterfactual Replays and Hybrid Wizard-of-Oz](https://mlsanity.com/paper/2510.06872/) (2025)
- [Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems](https://mlsanity.com/paper/2605.16278/) (2026)
- [Situated, Dynamic, and Subjective: Envisioning the Design of Theory-of-Mind-Enabled Everyday AI with Industry Practitioners](https://mlsanity.com/paper/2602.11342/) (2026)
- [LitPivot: Developing Well-Situated Research Ideas Through Dynamic Contextualization and Critique within the Literature Landscape](https://mlsanity.com/paper/2604.02600/) (2026)
- [Exploring the Innovation Opportunities for Pre-trained Models](https://mlsanity.com/paper/2505.15790/) (2025)
- [PolicyPad: Collaborative Prototyping of LLM Policies](https://mlsanity.com/paper/2509.19680/) (2025)
- [Emergent, not Immanent: A Baradian Reading of Explainable AI](https://mlsanity.com/paper/2601.15029/) (2026)
- ["So There's a Catch-22 Here": How Early Adopters Who Build Multi-Agent LLM Systems Conceptualize Transparency](https://mlsanity.com/paper/2606.08323/) (2026)
- [What We Know about Responsible AI Practices in Industry: A Half Decade of Empirical Research](https://mlsanity.com/paper/2608.10431/) (2026)

## Abstract (arXiv)

Despite the widespread use of artificial intelligence (AI), designing user experiences (UX) for AI-powered systems remains challenging. UX designers face hurdles understanding AI technologies, such as pre-trained language models, as design materials. This limits their ability to ideate and make decisions about whether, where, and how to use AI. To address this problem, we bridge the literature on AI design and AI transparency to explore whether and how frameworks for transparent model reporting can support design ideation with pre-trained models. By interviewing 23 UX practitioners, we find that practitioners frequently work with pre-trained models, but lack support for UX-led ideation. Through a scenario-based design task, we identify common goals that designers seek model understanding for and pinpoint their model transparency information needs. Our study highlights the pivotal role that UX designers can play in Responsible AI and calls for supporting their understanding of AI limitations through model transparency and interrogation.