mlsanityData to 14 Oct 2026

Human-on-the-Loop Orchestration for AI-Assisted Legal Discovery

arXiv 2606.19812 ↗First posted 18 Jun 2026cs.AI cs.LGcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Autonomous Large Language Model (LLM) agents are increasingly deployed in electronic discovery (e-discovery), where compounding errors across multi-step reasoning chains can constitute legal malpractice. Unlike single-turn retrieval, agentic workflows operating over privileged document corpora exhibit a class of failure we term "trajectory collapse": an early misclassification silently propagates, rendering an entire privilege review invalid. This paper makes three contributions. First, we propose a structured taxonomy of agentic failures in legal information retrieval, organized by functional stage. Second, we introduce a four-layer verification architecture -- spanning planning, reasoning, execution, and uncertainty quantification -- designed to intercept these failures before they compound. Third, we present a preliminary simulation study on a synthetic e-discovery corpus that demonstrates how mandatory Human-on-the-Loop (HOTL) escalation thresholds reduce privilege-waiver risk relative to fully autonomous baselines. Our results suggest that calibrated uncertainty thresholds can reduce privilege-waiver risk by up to 61% versus fully autonomous deployment, while routing fewer than one quarter of documents to attorney review.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 19 of its 20 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

19 of 20 references.

References · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  7. 2017Selective Classification for Deep Neural Networkscited by 852
  8. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  9. 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 183
  10. 2025SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attributioncited by 73
  11. 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedbackcited by 72
  12. 2024ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Modelscited by 25
7 more
  1. 2025Fault-Tolerant Sandboxing for AI Coding Agents: A Transactional Approach to Safe Autonomous Executioncited by 9
  2. 2026Multi-Agent Video Recommenders: Evolution, Patterns, and Open Challengescited by 6
  3. 2026TAPE: Tool-Guided Adaptive Planning and Constrained Execution in Language Model Agentscited by 4
  4. 2026KAIJU: An Executive Kernel for Intent-Gated Execution of LLM Agentscited by 2
  5. 2026Beyond Fluency: Toward Reliable Trajectories in Agentic IRcited by 2
  6. 2025ORCHID: Orchestrated Retrieval-Augmented Classification with Human-in-the-Loop Intelligent Decision-Making for High-Risk Propertycited by 1
  7. 2026STCALIR: Semi-Synthetic Test Collection for Algerian Legal Information Retrievalcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LawThinker: A Deep Research Legal Agent in Dynamic Environments
  2. 2026LawCompass: Navigating from Legal QA to Multi-Agent Deep Research with Grounded Evidence
  3. 2026Can LLMs Time Travel? Enhancing Temporal Consistency in Legal Agentic Search through Reinforcement Learning
  4. 2026Legal-R1: Agentic Retrieval for Evidence-Based Legal Reasoning
  5. 2026Beyond Probabilistic Similarity: Structural, Temporal, and Causal Limitations of Retrieval-Augmented Generation in the Legal Domain
  6. 2026From Naive RAG to Deep Agentic Retrieval: An Evolving Context Engineering Pipeline for Regulatory Compliance
  7. 2026LLM Agents in Law: Taxonomy, Applications, and Challenges
  8. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs