mlsanityData to 7 Oct 2026

Interpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation

arXiv 2109.10149 ↗First posted 21 Sep 2021cs.HC cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Feedback in creativity support tools can help crowdworkers to improve their ideations. However, current feedback methods require human assessment from facilitators or peers. This is not scalable to large crowds. We propose Interpretable Directed Diversity to automatically predict ideation quality and diversity scores, and provide AI explanations - Attribution, Contrastive Attribution, and Counterfactual Suggestions - to feedback on why ideations were scored (low), and how to get higher scores. These explanations provide multi-faceted feedback as users iteratively improve their ideations. We conducted formative and controlled user studies to understand the usage and usefulness of explanations to improve ideation diversity and quality. Users appreciated that explanation feedback helped focus their efforts and provided directions for improvement. This resulted in explanations improving diversity compared to no feedback or feedback with scores only. Hence, our approach opens opportunities for explainable AI towards scalable and rich feedback for iterative crowd ideation and creativity support tools.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 22 of its 106 references that are in the map. First posted on arXiv 21 Sep 2021.

Builds on

22 of 106 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  7. 2019Language Models as Knowledge Bases?cited by 1,922
  8. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  9. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  10. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
  11. 2018Universal Sentence Encodercited by 485
  12. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
10 more
  1. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  2. 2019Human-Centered Tools for Coping with Imperfect Algorithms during Medical Decision-Makingcited by 109
  3. 2018Building a Conversational Agent Overnight with Dialogue Self-Playcited by 70
  4. 2017Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanationscited by 37
  5. 2021Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideationcited by 23
  6. 2020Leveraging Rationales to Improve Human Task Performancecited by 20
  7. 2021Show or Suppress? Managing Input Uncertainty in Machine Learning Model Explanationscited by 15
  8. 2020Spinneret: Aiding Creative Ideation through Non-Obvious Concept Associationscited by 3
  9. 2021Hardhats and Bungaloos: Comparing Crowdsourced Design Feedback with Peer Design Feedback in the Classroomcited by 3
  10. 2020Supporting Creative Work with Crowd Feedback Systemscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 2; 2025: 4; 2026: 1.

12 papers.

  1. 2022Towards Human-centered Explainable AI: A Survey of User Studies for Model Explanationscited by 84
  2. 2023RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressionscited by 43
  3. 2023ConvXAI: Delivering Heterogeneous AI Explanations via Conversations to Support Human-AI Scientific Writingcited by 31
  4. 2023ScatterShot: Interactive In-context Example Curation for Text Transformationcited by 20
  5. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 15
  6. 2025Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Processcited by 14
  7. 2024Incremental XAI: Memorable Understanding of AI with Incremental Explanationscited by 8
  8. 2025GenTune: Toward Traceable Prompts to Improve Controllability of Image Refinement in Environment Designcited by 5
  9. 2023Diagrammatization and Abduction to Improve AI Interpretability With Domain-Aligned Explanations for Medical Diagnosiscited by 4
  10. 2025Redefining Research Crowdsourcing: Incorporating Human Feedback with LLM-Powered Digital Twinscited by 3
  11. 2025On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomycited by 2
  12. 2026The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotationcited by 1