Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMs
Not on the map. Why?
Abstract (arXiv)
A local specialist LLM, fine-tuned with reinforcement learning from verifiable rewards (RLVR) on operator-local data, is installed in a regulated organization with per-deployment error budget α. The operator needs a safety certificate for this deployment's stream at every round: no pooling across deployments, no waiting for a long-run average. Existing wrappers cannot deliver this on adaptive, online-updated streams: offline conformal-risk methods require exchangeability; online-conformal methods bound only long-run averages; non-exchangeable extensions are marginally valid; and the closest anytime wrapper, A-RCPS, controls marginal rather than selective risk. Using a (test statistic, validity guarantee, deployment rule) framework, we identify one empty cell forced by deployment requirements: e-process per threshold, selective risk, anytime-pathwise validity, max-certified-threshold rule. Conformal Selective Acting (CSA) fills it as a per-round wrapper maintaining a Ville-type e-process per threshold on a Bonferroni grid, evaluated against the RLVR filtration. Under predictable updates and isotonic-calibrated monotone risk we prove (i) an anytime-pathwise selective-risk bound RTact≤α+O(NT-1/2), (ii) rate-optimal certification matching Theta(barη-2log(1/δ)), and (iii) a horizon-independent release-rate gap. Across eight specialist benchmarks (480 streams), sixteen adversarial distribution-shift cells (160 streams), and five live Expert-Iteration RLVR cells with online LoRA over four base models in three architecture families (10,300 rounds), CSA is the only method among ten compared that satisfies pathwise validity and non-refusing deployment on every cell. We do not propose a new LLM, training algorithm, or policy class; CSA is the deployment-side complement, orthogonal to the model, for operators who cannot use a frontier API.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
32 of 40 references.
References · 32
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
- 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
- 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
20 more
- 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
- 2022Conformal prediction beyond exchangeabilitycited by 355
- 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
- 2022Conformal Risk Controlcited by 299
- 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
- 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 271
- 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 252
- 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
- 2020Estimating means of bounded random variables by bettingcited by 169
- 2018Lessons from Natural Language Inference in the Clinical Domaincited by 151
- 2024Language Models with Conformal Factuality Guaranteescited by 131
- 2019HEAD-QA: A Healthcare Dataset for Complex Reasoningcited by 101
- 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 83
- 2023Improved Online Conformal Prediction via Strongly Adaptive Online Learningcited by 78
- 2024Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approachescited by 52
- 2024Active, anytime-valid risk controlling prediction setscited by 14
- 2025Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Modelscited by 12
- 2019Online Control of the False Coverage Rate and False Sign Ratecited by 11
- 2025Fleming-R1: Toward Expert-Level Medical Reasoning via Reinforcement Learningcited by 10
- 2024CAP: A General Algorithm for Online Selective Conformal Prediction with FCR Controlcited by 9
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026ToolChain-CRC: Conformal Risk Control for Agentic AI Under Retrieval and Tool-Use Driftcited by 6
- 2026When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generationcited by 4
- 2026Certified Adaptive Refresh: Anytime-Valid Monitoring for Federated Conformal RAGcited by 2
- 2026Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Callscited by 1
- 2026Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agentscited by 1
- 2026When Can Fraud Operations Authorize Automation? A Decision-Support Framework for Fresh Audit Evidence and Review Workloadcited by 0
- 2026Non-Degenerate Risk Certification for Automated Security Decisions: A Decision-Contract Theory with ATT&CK-Aligned Triage as a Worked Instancecited by 0
- 2026Finding Blind Spots in AppWorld and WorkArena Task Verifierscited by 0
- 2026Finding Blind Spots in AppWorld and WorkArena Task Verifiers6 Oct 2026
- 2026Non-Degenerate Risk Certification for Automated Security Decisions: A Decision-Contract Theory with ATT&CK-Aligned Triage as a Worked Instance12 Aug 2026
- 2026When Can Fraud Operations Authorize Automation? A Decision-Support Framework for Fresh Audit Evidence and Review Workload9 Aug 2026
- 2026Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agents4 Aug 2026
- 2026Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls27 Jul 2026
- 2026When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generation27 Jun 2026
- 2026ToolChain-CRC: Conformal Risk Control for Agentic AI Under Retrieval and Tool-Use Drift16 Jun 2026
- 2026Certified Adaptive Refresh: Anytime-Valid Monitoring for Federated Conformal RAG27 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewards
- 2026FSPO: Policy-Consistent Risk and Pareto-Feasible Control for Budgeted LLM RL Post-Training
- 2026MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents
- 2026Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment
- 2025Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models
- 2025Breaking the Safety-Capability Tradeoff: Reinforcement Learning with Verifiable Rewards Maintains Safety Guardrails in LLMs
- 2025Verification-Guided Falsification for Safe RL via Explainable Abstraction and Risk-Aware Exploration
- 2026Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safety