# Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservables arXiv: https://arxiv.org/abs/2207.13834 ยท HTML: https://mlsanity.com/paper/2207.13834/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 56 references that are in the map. - First posted on arXiv 28 Jul 2022. ## Builds on (most cited first, 20 of 20) - [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020) - [Manipulating and Measuring Model Interpretability](https://mlsanity.com/paper/1802.07810/) (2018) - [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017) - [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019) - [Learning to Complement Humans](https://mlsanity.com/paper/2005.00582/) (2020) - [Building machines that adapt and compute like brains](https://mlsanity.com/paper/1711.04203/) (2017) - [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020) - [Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes](https://mlsanity.com/paper/1904.09612/) (2019) - [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022) - [Human-AI Collaboration with Bandit Feedback](https://mlsanity.com/paper/2105.10614/) (2021) - ["Brilliant AI Doctor" in Rural China: Tensions and Challenges in AI-Powered CDSS Deployment](https://mlsanity.com/paper/2101.01524/) (2021) - [A Human-Centered Review of the Algorithms used within the U.S. Child Welfare System](https://mlsanity.com/paper/2003.03541/) (2020) - [Algorithms and Decision-Making in the Public Sector](https://mlsanity.com/paper/2106.03673/) (2021) - [Investigating Human + Machine Complementarity for Recidivism Predictions](https://mlsanity.com/paper/1808.09123/) (2018) - [Designing for human-AI complementarity in K-12 education](https://mlsanity.com/paper/2104.01266/) (2021) - [Leveraging Expert Consistency to Improve Algorithmic Decision Support](https://mlsanity.com/paper/2101.09648/) (2021) - [On the Effect of Information Asymmetry in Human-AI Teams](https://mlsanity.com/paper/2205.01467/) (2022) - [Fairer and more accurate, but for whom?](https://mlsanity.com/paper/1707.00046/) (2017) - [The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies](https://mlsanity.com/paper/2109.01443/) (2021) - [The TA Framework: Designing Real-time Teaching Augmentation for K-12 Classrooms](https://mlsanity.com/paper/2001.02985/) (2020) ## Built on by (most cited first, 19 of 19) - [Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making](https://mlsanity.com/paper/2209.11812/) (2022) - [Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making](https://mlsanity.com/paper/2302.06503/) (2023) - [Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review](https://mlsanity.com/paper/2310.19778/) (2023) - [AI, Help Me Thinkunicodex2014but for Myself: Assisting People in Complex Decision-Making by Providing Different Kinds of Cognitive Support](https://mlsanity.com/paper/2504.06771/) (2025) - [A Systematic Literature Review of Human-Centered, Ethical, and Responsible AI](https://mlsanity.com/paper/2302.05284/) (2023) - [Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge](https://mlsanity.com/paper/2308.15700/) (2023) - [Explaining and Improving Information Complementarities in Multi-Agent Decision-making](https://mlsanity.com/paper/2502.06152/) (2025) - [Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information](https://mlsanity.com/paper/2401.04729/) (2024) - [A Survey of AI Reliance](https://mlsanity.com/paper/2408.03948/) (2024) - [Confounding-Robust Policy Improvement with Human-AI Teams](https://mlsanity.com/paper/2310.08824/) (2023) - [Towards Uncertainty Aware Task Delegation and Human-AI Collaborative Decision-Making](https://mlsanity.com/paper/2505.18066/) (2025) - [Perils of Label Indeterminacy: A Case Study on Prediction of Neurological Recovery After Cardiac Arrest](https://mlsanity.com/paper/2504.04243/) (2025) - [ComplLLM: Fine-tuning LLMs to Discover Complementary Signals for Decision-making](https://mlsanity.com/paper/2602.19458/) (2026) - [A Risk Taxonomy and Reflection Tool for Large Language Model Adoption in Public Health](https://mlsanity.com/paper/2411.02594/) (2024) - [AI LEGO: Scaffolding Cross-Functional Collaboration in Industrial Responsible AI Practices during Early Design Stages](https://mlsanity.com/paper/2505.10300/) (2025) - [From Accuracy to Readiness: Metrics and Benchmarks for Human-AI Decision-Making](https://mlsanity.com/paper/2603.18895/) (2026) - [Interactive Example-based Explanations to Improve Health Professionals' Onboarding with AI for Human-AI Collaborative Decision Making](https://mlsanity.com/paper/2409.15814/) (2024) - [Exploring Human-AI Complementarity in CPS Diagnosis Using Unimodal and Multimodal BERT Models](https://mlsanity.com/paper/2507.14579/) (2025) - [RuleEdit: Failure-Guided Human-AI Model Editing with Prospective Impact Preview](https://mlsanity.com/paper/2606.00011/) (2026) ## Abstract (arXiv) In many real world contexts, successful human-AI collaboration requires humans to productively integrate complementary sources of information into AI-informed decisions. However, in practice human decision-makers often lack understanding of what information an AI model has access to in relation to themselves. There are few available guidelines regarding how to effectively communicate about unobservables: features that may influence the outcome, but which are unavailable to the model. In this work, we conducted an online experiment to understand whether and how explicitly communicating potentially relevant unobservables influences how people integrate model outputs and unobservables when making predictions. Our findings indicate that presenting prompts about unobservables can change how humans integrate model outputs and unobservables, but do not necessarily lead to improved performance. Furthermore, the impacts of these prompts can vary depending on decision-makers' prior domain expertise. We conclude by discussing implications for future research and design of AI-based decision support tools.