mlsanityData to 7 Oct 2026

Human-AI Complementarity: A Goal for Amplified Oversight

arXiv 2510.26518 ↗First posted 30 Oct 2025cs.AI cs.HCcited by 3

Not on the map. Why?

Abstract (arXiv)

Human feedback is critical for aligning AI systems to human values. As AI capabilities improve and AI is used to tackle more challenging tasks, verifying quality and safety becomes increasingly challenging. This paper explores how we can leverage AI to improve the quality of human oversight. We focus on an important safety problem that is already challenging for humans: fact-verification of AI outputs. We find that combining AI ratings and human ratings based on AI rater confidence is better than relying on either alone. Giving humans an AI fact-verification assistant further improves their accuracy, but the type of assistance matters. Displaying AI explanation, confidence, and labels leads to over-reliance, but just showing search results and evidence fosters more appropriate trust. These results have implications for Amplified Oversight -- the challenge of combining humans and AI to supervise AI systems even as they surpass human expert performance.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 75 references that are in the map. First posted on arXiv 30 Oct 2025.

Builds on

39 of 75 references.

References · 39

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2016Concrete Problems in AI Safetycited by 1,582
  4. 2023Towards Understanding Sycophancy in Language Modelscited by 988
  5. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  6. 2020Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performancecited by 396
  7. 2018AI safety via debatecited by 384
  8. 2022Self-critiquing models for assisting human evaluatorscited by 372
  9. 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
  10. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  11. 2024Frontier Models are Capable of In-context Schemingcited by 252
  12. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
27 more
  1. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 228
  2. 2018On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detectioncited by 159
  3. 2024Long-form factuality in large language modelscited by 146
  4. 2018Supervising strong learners by amplifying weak expertscited by 139
  5. 2020Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systemscited by 138
  6. 2024LLM Critics Help Catch LLM Bugscited by 138
  7. 2021To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-makingcited by 118
  8. 2022Explanations Can Reduce Overreliance on AI Systems During Decision-Makingcited by 111
  9. 2019The Algorithmic Automation Problem: Prediction, Triage, and Human Effortcited by 101
  10. 2024When combinations of humans and AI are useful: A systematic review and meta-analysiscited by 100
  11. 2024On scalable oversight with weak LLMs judging strong LLMscited by 94
  12. 2023Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanationscited by 80
  13. 2023Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Makingcited by 67
  14. 2024"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trustcited by 67
  15. 2023Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision supportcited by 65
  16. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  17. 2025Gradual Disempowerment: Systemic Existential Risks from Incremental AI Developmentcited by 42
  18. 2025An Approach to Technical AGI Safety and Securitycited by 42
  19. 2023Debate Helps Supervise Unreliable Expertscited by 39
  20. 2021Role of Human-AI Interaction in Selective Predictioncited by 33
  21. 2023ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principlescited by 33
  22. 2023Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfactioncited by 32
  23. 2024"Are You Really Sure?" Understanding the Effects of Human Self-Confidence Calibration in AI-Assisted Decision Makingcited by 26
  24. 2024A Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregationcited by 23
  25. 2022Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questionscited by 22
  26. 2022Two-Turn Debate Doesn't Help Humans Answer Hard Reading Comprehension Questionscited by 16
  27. 2025Fine-Grained Appropriate Reliance: Human-AI Collaboration with a Multi-Step Transparent Decision Workflow for Complex Task Decompositioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Can AI mediation improve democratic deliberation?cited by 0
  2. 2026Building Intelligent User Interfaces for Human-AI Alignmentcited by 0
  3. 2026Toward Human-AI Complementarity Across Diverse Taskscited by 0