# Human-in-the-loop or AI-in-the-loop? Automate or Collaborate?

arXiv: https://arxiv.org/abs/2412.14232 · HTML: https://mlsanity.com/paper/2412.14232/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2024.

## Built on by (most cited first, 23 of 23)

- [LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey](https://mlsanity.com/paper/2505.00753/) (2025)
- [AutoResearchClaw: Self-Reinforcing Autonomous Research with Human-AI Collaboration](https://mlsanity.com/paper/2605.20025/) (2026)
- [A Call for Collaborative Intelligence: Why Human-Agent Systems Should Precede AI Autonomy](https://mlsanity.com/paper/2506.09420/) (2025)
- [Formalising Human-in-the-Loop: Computational Reductions, Failure Modes, and Legal-Moral Responsibility](https://mlsanity.com/paper/2505.10426/) (2025)
- [Explainable Anomaly Detection for Industrial IoT Data Streams](https://mlsanity.com/paper/2512.08885/) (2025)
- [Toward Safe and Responsible AI Agents: A Three-Pillar Model for Transparency, Accountability, and Trustworthiness](https://mlsanity.com/paper/2601.06223/) (2026)
- [Interaction-Driven Browsing: A Human-in-the-Loop Conceptual Framework Informed by Human Web Browsing for Browser-Using Agents](https://mlsanity.com/paper/2509.12049/) (2025)
- [SOCIA-nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generation](https://mlsanity.com/paper/2505.12006/) (2025)
- [A Decoupled Human-in-the-Loop System for Controlled Autonomy in Agentic Workflows](https://mlsanity.com/paper/2604.23049/) (2026)
- [Measuring Biological Capabilities and Risks of AI Agents](https://mlsanity.com/paper/2606.19899/) (2026)
- [Recalibrating the Compass: Integrating Large Language Models into Classical Research Methods](https://mlsanity.com/paper/2505.19402/) (2025)
- [Human Empathy as Encoder: AI-Assisted Depression Assessment in Special Education](https://mlsanity.com/paper/2505.23631/) (2025)
- [Conveying Imagistic Thinking in Traditional Chinese Medicine Translation: A Prompt Engineering and LLM-Based Evaluation Framework](https://mlsanity.com/paper/2511.23059/) (2025)
- [Conveying Imagistic Thinking in Traditional Chinese Medicine Translation: A Prompt Engineering and LLM-Based Evaluation Framework](https://mlsanity.com/paper/2512.01198/) (2025)
- [GRASP: Graph Reasoning Agents for Systems Pharmacology with Human-in-the-Loop](https://mlsanity.com/paper/2512.05502/) (2025)
- [MARBLE: Multi-Agent Reasoning for Bioinformatics Learning and Evolution](https://mlsanity.com/paper/2601.14349/) (2026)
- [Inclusive AI for Group Interactions: Predicting Gaze-Direction Behaviors in People with Intellectual and Developmental Disabilities](https://mlsanity.com/paper/2603.14460/) (2026)
- [GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning](https://mlsanity.com/paper/2604.16871/) (2026)
- [AI to Learn 2.0: A Deliverable-Oriented Governance Framework and Maturity Rubric for Opaque AI in Learning-Intensive Domains](https://mlsanity.com/paper/2604.19751/) (2026)
- [Humanwashing -- It Should Leave You Feeling Dirty](https://mlsanity.com/paper/2605.13723/) (2026)
- [Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction](https://mlsanity.com/paper/2608.01792/) (2026)
- [A Neurosymbolic Approach for Constructing Planning Domain Models from Clinical Narratives](https://mlsanity.com/paper/2608.21186/) (2026)
- [Collaborative Streaming Anomaly Detection with Interactive Explanations and Ensemble Consensus](https://mlsanity.com/paper/2609.23883/) (2026)

## Abstract (arXiv)

Human-in-the-loop (HIL) systems have emerged as a promising approach for combining the strengths of data-driven machine learning models with the contextual understanding of human experts. However, a deeper look into several of these systems reveals that calling them HIL would be a misnomer, as they are quite the opposite, namely AI-in-the-loop (AI2L) systems, where the human is in control of the system, while the AI is there to support the human. We argue that existing evaluation methods often overemphasize the machine (learning) component's performance, neglecting the human expert's critical role. Consequently, we propose an AI2L perspective, which recognizes that the human expert is an active participant in the system, significantly influencing its overall performance. By adopting an AI2L approach, we can develop more comprehensive systems that faithfully model the intricate interplay between the human and machine components, leading to more effective and robust AI systems.