mlsanityData to 7 Oct 2026

Multi-Round Human-AI Collaboration with User-Specified Requirements

arXiv 2602.17646 ↗First posted 19 Feb 2026cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

As humans increasingly rely on multiround conversational AI for high stakes decisions, principled frameworks are needed to ensure such interactions reliably improve decision quality. We adopt a human centric view governed by two principles: counterfactual harm, ensuring the AI does not undermine human strengths, and complementarity, ensuring it adds value where the human is prone to err. We formalize these concepts via user defined rules, allowing users to specify exactly what harm and complementarity mean for their specific task. We then introduce an online, distribution free algorithm with finite sample guarantees that enforces the user-specified constraints over the collaboration dynamics. We evaluate our framework across two interactive settings: LLM simulated collaboration on a medical diagnostic task and a human crowdsourcing study on a pictorial reasoning task. We show that our online procedure maintains prescribed counterfactual harm and complementarity violation rates even under nonstationary interaction dynamics. Moreover, tightening or loosening these constraints produces predictable shifts in downstream human accuracy, confirming that the two principles serve as practical levers for steering multi-round collaboration toward better decision quality without the need to model or constrain human behavior.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 51 of its 72 references that are in the map. First posted on arXiv 19 Feb 2026.

Builds on

51 of 72 references.

References · 51

  1. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  2. 2019Conformalized Quantile Regressioncited by 620
  3. 2020Classification with Valid and Adaptive Coveragecited by 390
  4. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  5. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 360
  6. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  7. 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
  8. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 228
  9. 2020Learning to Complement Humanscited by 123
  10. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 123
  11. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  12. 2019The Algorithmic Automation Problem: Prediction, Triage, and Human Effortcited by 101
38 more
  1. 2024When combinations of humans and AI are useful: A systematic review and meta-analysiscited by 100
  2. 2021Algorithmic Monoculture and Social Welfarecited by 98
  3. 2023Improved Online Conformal Prediction via Strongly Adaptive Online Learningcited by 76
  4. 2020A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scorescited by 74
  5. 2021Differentiable Learning Under Triagecited by 74
  6. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
  7. 2021Towards Unbiased and Accurate Deferral to Multiple Expertscited by 71
  8. 2022Calibrated Learning to Defer with One-vs-All Classifierscited by 68
  9. 2022Improving Expert Predictions with Conformal Predictioncited by 54
  10. 2024Online conformal prediction with decaying step sizescited by 53
  11. 2022Sample Efficient Learning of Predictors that Complement Humanscited by 46
  12. 2023Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictionscited by 46
  13. 2025Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agentscited by 37
  14. 2022On the Utility of Prediction Sets in Human-AI Teamscited by 34
  15. 2022Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imagingcited by 32
  16. 2023How Bad is Top-K Recommendation under Competing Content Creators?cited by 27
  17. 2024Formal Verification and Control with Conformal Predictioncited by 26
  18. 2023Learning to Defer with Limited Expert Predictionscited by 25
  19. 2023When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programmingcited by 25
  20. 2024Towards Human-AI Complementarity with Prediction Setscited by 25
  21. 2018Incentivizing Exploration with Selective Data Disclosurecited by 24
  22. 2022A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementaritycited by 22
  23. 2019The Assistive Multi-Armed Banditcited by 19
  24. 2022Improving Screening Processes via Calibrated Subset Selectioncited by 16
  25. 2025Collaborative Prediction: Tractable Information Aggregation via Agreementcited by 14
  26. 2025The Relationship between No-Regret Learning and Online Conformal Predictioncited by 13
  27. 2024Tractable Agreement Protocolscited by 12
  28. 2022Algorithmic Assistance with Recommendation-Dependent Preferencescited by 11
  29. 2023Towards Modeling and Influencing the Dynamics of Human Learningcited by 11
  30. 2024Utility-Directed Conformal Prediction: A Decision-Aware Framework for Actionable Uncertainty Quantificationcited by 11
  31. 2017Fairness Incentives for Myopic Agentscited by 10
  32. 2020A Bandit Model for Human-Machine Decision Making with Private Information and Opacitycited by 10
  33. 2024Conformal Risk Minimization with Variance Reductioncited by 10
  34. 2022Diversified Recommendations for Agents with Adaptive Preferencescited by 9
  35. 2024Controlling Counterfactual Harm in Decision Support Systems Based on Prediction Setscited by 9
  36. 2025Towards Human-AI Complementarity in Matching Taskscited by 7
  37. 2025Conformal Prediction Beyond the Seen: A Missing Mass Perspective for Uncertainty Quantification in Generative Modelscited by 6
  38. 2025Human-AI Collaborative Uncertainty Quantificationcited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Conformal Risk-Averse Decision Making with Action Conditional Guaranteecited by 1
  2. 2026Strategic Decision Support for AI Agentscited by 1