mlsanityData to 14 Oct 2026

More Rejective, Not More Discriminative: The Unit of Verification in Pre-Execution LLM Oversight

arXiv 2608.23941 ↗First posted 25 Aug 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Pre-execution oversight is core to trusted monitoring in AI control: a fallible LLM monitor vets planned actions before irreversible execution. Over-blocking forfeits usefulness and pressures deployers to disable it. Every protocol must fix a unit of verification: how many actions one call reviews. Existing designs take the unit as given; its effect on fallible monitors is unmeasured. Natural traces cannot isolate it: review length co-varies with error type and position. Catch alone misleads: rejecting everything catches everything. Measuring this needs boundary variation alone and a matched clean control. We introduce the twin-prefix framework, which supplies both. Each gold plan yields a prefix with one injected, environment-accepted error and a clean twin differing in one write. Judging each pair at five nested lengths ties verdict changes to the unit alone. Discrimination is scored by pre-registered informedness, catch minus false rejection. Longer review raises catch; false rejection climbs in lockstep. Informedness peaks at one or two actions for all six judges in both domains: longer windows make zero-shot monitors more rejective, not more discriminative. Replaying withheld observations traces the failure largely to observation deprivation. Safety cases should state the unit and co-report the clean series. Our framework is the first controlled, pre-registered instrument for this choice and never reads catch alone. Our calibrated short unit recovers up to 0.95 informedness over eight-action review, and no tested label-blind policy consistently beats it.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 18 of its 34 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

18 of 34 references.

References · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  4. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 425
  5. 2020Selective Question Answering under Domain Shiftcited by 238
  6. 2025Ctrl-Z: Controlling AI Agents via Resamplingcited by 44
  7. 2025A sketch of an AI control safety casecited by 39
  8. 2026AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agentscited by 31
  9. 2025Adaptive Attacks on Trusted Monitors Subvert AI Control Protocolscited by 30
  10. 2026ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agentscited by 14
  11. 2026The Verifier Tax: Horizon Dependent Safety Success Tradeoffs in Tool Using LLM Agentscited by 11
  12. 2025Control Tax: The Price of Keeping AI in Checkcited by 10
6 more
  1. 2026When to Trust the Cheap Check: Weak and Strong Verification for Reasoningcited by 7
  2. 2025Rethinking Optimal Verification Granularity for Compute-Efficient Test-Time Scalingcited by 5
  3. 2026The Hidden Signal of Verifier Strictness: Controlling and Improving Step-Wise Verification via Selective Latent Steeringcited by 5
  4. 2025FindTheFlaws: Annotated Errors for Detecting Flawed Reasoning and Scalable Oversight Researchcited by 3
  5. 2026Granularity-Regulated Adaptive Computational Efficiency for Optimal Verification in Test-Time Scalingcited by 1
  6. 2026Certified Speculative Execution for Untrusted AI Agentscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.