mlsanityData to 14 Oct 2026

Strategic Decision Support for AI Agents

arXiv 2606.12587 ↗First posted 10 Jun 2026cs.AI cs.HCcited by 1

Not on the map. Why?

Abstract (arXiv)

Traditionally, decision support studies how humans use machine learning models to make better decisions. In modern agentic systems, this division of roles is increasingly reversed: AI agents act on behalf of users, while humans and tools becomes support mechanisms around them. This role reversal brings reliability concerns to the forefront, since agentic errors can be consequential and agent behavior must remain aligned with human goals and constraints. Departing from the classical view of decision support, we revisit its two basic principles, the cost--value tradeoff of seeking support and the role of uncertainty quantification, in a setting where AI agents are the central actors. We propose a framework for strategic decision support for AI agents through an optimization problem that minimizes support usage subject to controlling a counterfactual missed-support error: the probability that the agent acts alone on instances where support would have materially improved its output. At the population level, we show that the optimal policy is a threshold rule on the value of support. Building on this structure, we develop an online algorithm that adaptively thresholds such a score and uses randomized exploration to control missed-support error without distributional assumptions. We further introduce a calibration-on-the-fly method that reduces unnecessary support calls online. We instantiate this framework across diverse scenarios, including information gathering, human--AI collaboration, and tool use, showing how each can be modeled through the same strategic decision-support lens. Experiments across these settings show that our method reliably controls the target error while substantially reducing support usage in practice.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 73 of its 83 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

73 of 83 references.

References · 73

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  5. 2017Selective Classification for Deep Neural Networkscited by 852
  6. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  7. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  8. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 687
  9. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  10. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  11. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  12. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
38 more
  1. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  2. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  3. 2020Consistent Estimators for Learning to Defer to an Expertcited by 271
  4. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 233
  5. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  6. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
  7. 2024Language Models with Conformal Factuality Guaranteescited by 131
  8. 2020Learning to Complement Humanscited by 124
  9. 2024To Believe or Not to Believe Your LLMcited by 120
  10. 2023Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaborationcited by 116
  11. 2024When combinations of humans and AI are useful: A systematic review and meta-analysiscited by 106
  12. 2024STaR-GATE: Teaching Language Models to Ask Clarifying Questionscited by 104
  13. 2019The Algorithmic Automation Problem: Prediction, Triage, and Human Effortcited by 101
  14. 2021Algorithmic Monoculture and Social Welfarecited by 100
  15. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 100
  16. 2023Eliciting Human Preferences with Language Modelscited by 92
  17. 2024Large language model validity via enhanced conformal prediction methodscited by 92
  18. 2023Conformal PID Control for Time Series Predictioncited by 88
  19. 2021Differentiable Learning Under Triagecited by 76
  20. 2024Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approachcited by 76
  21. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 76
  22. 2021Towards Unbiased and Accurate Deferral to Multiple Expertscited by 72
  23. 2022Calibrated Learning to Defer with One-vs-All Classifierscited by 69
  24. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 69
  25. 2022Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensemblescited by 60
  26. 2022Improving Expert Predictions with Conformal Predictioncited by 54
  27. 2022Sample Efficient Learning of Predictors that Complement Humanscited by 46
  28. 2020Classification Under Human Assistancecited by 42
  29. 2024Learning to Ask: When LLM Agents Meet Unclear Instructioncited by 42
  30. 2025When2Call: When (not) to Call Toolscited by 42
  31. 2019Regression Under Human Assistancecited by 40
  32. 2022On the Utility of Prediction Sets in Human-AI Teamscited by 34
  33. 2024Towards Uncertainty-Aware Language Agentcited by 32
  34. 2023Learning to Defer with Limited Expert Predictionscited by 25
  35. 2024Towards Human-AI Complementarity with Prediction Setscited by 25
  36. 2022A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementaritycited by 23
  37. 2024Non-Exchangeable Conformal Language Generation with Nearest Neighborscited by 21
  38. 2025Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Modelscited by 21

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Risk-Aware Online Conformal State Probingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Human Performance with Value-Aware Interventions: A Case Study in Chess
  2. 2024Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning
  3. 2023Learning Complementary Policies for Human-AI Teams
  4. 2026Human-Centric Reflective Architecture for Human-AI Collaborative Decision-Making
  5. 2025Emergent Risk Awareness in Rational Agents under Resource Constraints
  6. 2026Supporting Calibrated Reliance in Human-AI Collaboration: Different Strategies for Different Tasks
  7. 2025Mathematical Framing for Different Agent Strategies
  8. 2025Teaching AI to Handle Exceptions: Supervised Fine-Tuning with Human-Aligned Judgment