mlsanityData to 14 Oct 2026

Improving Human-AI Collaboration With Descriptions of AI Behavior

arXiv 2301.06937 ↗First posted 6 Jan 2023cs.HC cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

People work with AI systems to improve their decision making, but often under- or over-rely on AI predictions and perform worse than they would have unassisted. To help people appropriately rely on AI aids, we propose showing them behavior descriptions, details of how AI systems perform on subgroups of instances. We tested the efficacy of behavior descriptions through user studies with 225 participants in three distinct domains: fake review detection, satellite image classification, and bird classification. We found that behavior descriptions can increase human-AI accuracy through two mechanisms: helping people identify AI failures and increasing people's reliance on the AI when it is more accurate. These findings highlight the importance of people's mental models in human-AI collaboration and show that informing people of high-level AI behaviors can significantly improve AI-assisted decision making.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 27 of its 71 references that are in the map. First posted on arXiv 6 Jan 2023.

Builds on

27 of 71 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  3. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  5. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  6. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
  7. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460
  8. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 399
  9. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  10. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  11. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  12. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
15 more
  1. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  2. 2019FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learningcited by 93
  3. 2020"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humanscited by 86
  4. 2020A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scorescited by 74
  5. 2018Automated Data Slicing for Model Validation:A Big data - AI Integration Approachcited by 70
  6. 2018Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failurecited by 49
  7. 2020Conceptual Metaphors Impact Perceptions of Human-AI Collaborationcited by 47
  8. 2022Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Supportcited by 39
  9. 2020Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Predictioncited by 36
  10. 2021Teaching Humans When To Defer to a Classifier via Exemplarscited by 29
  11. 2019Structure-Invariant Testing for Machine Translationcited by 28
  12. 2021Discovering and Validating AI Errors With Crowdsourced Failure Reportscited by 28
  13. 2021Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Developmentcited by 21
  14. 2020PulseSatellite: A tool using human-AI feedback loops for satellite image analysis in humanitarian contextscited by 4
  15. 2020Personalization in Human-AI Teams: Improving the Compatibility-Accuracy Tradeoffcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 12; 2025: 11; 2026: 8.

36 papers.

Sort:

By area:

  1. 2024When combinations of humans and AI are useful: A systematic review and meta-analysiscited by 106
  2. 2024Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Makingcited by 38
  3. 2023The Impact of Imperfect XAI on Human-AI Decision-Makingcited by 21
  4. 2023Effective Human-AI Teams via Learned Natural Language Rules and Onboardingcited by 20
  5. 2025When Your Reviewer is an LLM: Biases, Divergence, and Prompt Injection Risks in Peer Reviewcited by 19
  6. 2023Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic reviewcited by 18
  7. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 18
  8. 2024Perceptions of Sentient AI and Other Digital Minds: Evidence from the AI, Morality, and Sentience (AIMS) Surveycited by 15
  9. 2024Contrastive Explanations That Anticipate Human Misconceptions Can Improve Human Decision-Making Skillscited by 14
  10. 2024Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learningcited by 13
  11. 2023AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventionscited by 9
  12. 2024A No Free Lunch Theorem for Human-AI Collaborationcited by 8
24 more
  1. 2025Linguistic Comparison of AI- and Human-Written Responses to Online Mental Health Queriescited by 6
  2. 2025AI Agent Behavioral Sciencecited by 6
  3. 2023Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbookcited by 5
  4. 2024Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Informationcited by 5
  5. 2024A Survey of AI Reliancecited by 5
  6. 2024Towards certifiable AI in aviation: landscape, challenges, and opportunitiescited by 2
  7. 2025A Mathematical Framework for AI-Human Integration in Workcited by 2
  8. 2025Reduced AI Acceptance After the Generative AI Boom: Evidence From a Two-Wave Survey Studycited by 2
  9. 2026The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotationcited by 2
  10. 2024A Risk Taxonomy and Reflection Tool for Large Language Model Adoption in Public Healthcited by 1
  11. 2024Generative AI in Medicinecited by 1
  12. 2025When Trust Collides: Decoding Human-LLM Cooperation Dynamics through the Prisoner's Dilemmacited by 1
  13. 2025Achieving Trustworthy Real-Time Decision Support Systems with Low-Latency Interpretable AI Modelscited by 1
  14. 2025AI That Helps Us Help Each Other: A Proactive System for Scaffolding Mentor-Novice Collaboration in Entrepreneurship Coachingcited by 1
  15. 2025Data-Prompt Co-Evolution: Growing Test Sets to Refine LLM Behaviorcited by 1
  16. 2025Teaching People LLM's Errors and Getting it Rightcited by 1
  17. 2026Human Control Is the Anchor, Not the Answer: Early Divergence of Oversight in Agentic AI Communitiescited by 1
  18. 2025AI-Powered Annotation Pipelines for Stabilizing Large Language Models: A Human-AI Synergy Approachcited by 0
  19. 2026Exploring Learners' Expectations and Engagement When Collaborating with Constructively Controversial Peer Agentscited by 0
  20. 2026When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoptioncited by 0
  21. 2026Intelligent Reasoning Cues: A Framework and Case Study of the Roles of AI Information in Complex Decisionscited by 0
  22. 2026The Timing Dependencies of Trust: Speed, Accuracy, and cBCI Neuro-Decoupling in Human-AI Teamscited by 0
  23. 2026From Prompts to Context: An Ontology-Driven Framework for Human-Generative AI Collaborationcited by 0
  24. 2026ASSEMCAD: Production-Ready CAD Assembly Generation from Natural Languagecited by 0