mlsanityData to 7 Oct 2026

StepCOPS: Closed-Testing Lower-Tail Certificates for Language-Model Policy Selection

arXiv 2609.29549 ↗First posted 26 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Post-training pipelines must select one language-model policy from many checkpoints, prompts, and decoding rules. Mean evaluator scores can conceal rare failures, whereas simultaneous candidate-wise confidence bounds can be unnecessarily conservative. We introduce StepCOPS, which uses an independent proposal split to nominate one lower-tail floor per candidate, exact binomial tests on a fresh certification split, and Holm's step-down procedure to certify a set of floors. With probability at least 1-δ, every certified floor, including the largest floor used for policy selection, is below its candidate's population lower α-quantile. This guarantee assumes i.i.d. evaluation units while allowing arbitrary within-unit dependence across candidates. Across 24 predeclared configurations and 11 benchmarks, StepCOPS obtains 96.4% selected-policy coverage over 500 paired trials, raises the certified floor by 1.5 points over both proposal-Bonferroni and exact COPS, remains 0.6 points below the large-reference jury oracle, and abstains in 2.4% of trials. Shadow-judge, benchmark-native, artifact, and leave-one-judge-out audits characterize the proxy boundary: the guarantee applies to the fixed jury score, not directly to human safety.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

19 of 31 references.

References · 19

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  4. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  5. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  6. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  9. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  10. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  11. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  12. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
7 more
  1. 2019Batch Policy Learning under Constraintscited by 242
  2. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  3. 2020Hyperparameter Selection for Offline Reinforcement Learningcited by 113
  4. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  5. 2021Active Offline Policy Selectioncited by 26
  6. 2025Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifactscited by 25
  7. 2025Conformal Tail Risk Control for Large Language Model Alignmentcited by 8

Built on by

Not cited yet.