mlsanityData to 14 Oct 2026

Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMs

arXiv 2605.20270 ↗First posted 18 May 2026cs.LG cs.AI stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

A local specialist LLM, fine-tuned with reinforcement learning from verifiable rewards (RLVR) on operator-local data, is installed in a regulated organization with per-deployment error budget α. The operator needs a safety certificate for this deployment's stream at every round: no pooling across deployments, no waiting for a long-run average. Existing wrappers cannot deliver this on adaptive, online-updated streams: offline conformal-risk methods require exchangeability; online-conformal methods bound only long-run averages; non-exchangeable extensions are marginally valid; and the closest anytime wrapper, A-RCPS, controls marginal rather than selective risk. Using a (test statistic, validity guarantee, deployment rule) framework, we identify one empty cell forced by deployment requirements: e-process per threshold, selective risk, anytime-pathwise validity, max-certified-threshold rule. Conformal Selective Acting (CSA) fills it as a per-round wrapper maintaining a Ville-type e-process per threshold on a Bonferroni grid, evaluated against the RLVR filtration. Under predictable updates and isotonic-calibrated monotone risk we prove (i) an anytime-pathwise selective-risk bound RTact≤α+O(NT-1/2), (ii) rate-optimal certification matching Theta(barη-2log(1/δ)), and (iii) a horizon-independent release-rate gap. Across eight specialist benchmarks (480 streams), sixteen adversarial distribution-shift cells (160 streams), and five live Expert-Iteration RLVR cells with online LoRA over four base models in three architecture families (10,300 rounds), CSA is the only method among ten compared that satisfies pathwise validity and non-refusing deployment on every cell. We do not propose a new LLM, training algorithm, or policy class; CSA is the deployment-side complement, orthogonal to the model, for operators who cannot use a frontier API.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

32 of 40 references.

References · 32

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  8. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  9. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  10. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  11. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  12. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 408
20 more
  1. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
  2. 2022Conformal prediction beyond exchangeabilitycited by 355
  3. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
  4. 2022Conformal Risk Controlcited by 299
  5. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
  6. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 271
  7. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 252
  8. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  9. 2020Estimating means of bounded random variables by bettingcited by 169
  10. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 151
  11. 2024Language Models with Conformal Factuality Guaranteescited by 131
  12. 2019HEAD-QA: A Healthcare Dataset for Complex Reasoningcited by 101
  13. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 83
  14. 2023Improved Online Conformal Prediction via Strongly Adaptive Online Learningcited by 78
  15. 2024Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approachescited by 52
  16. 2024Active, anytime-valid risk controlling prediction setscited by 14
  17. 2025Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Modelscited by 12
  18. 2019Online Control of the False Coverage Rate and False Sign Ratecited by 11
  19. 2025Fleming-R1: Toward Expert-Level Medical Reasoning via Reinforcement Learningcited by 10
  20. 2024CAP: A General Algorithm for Online Selective Conformal Prediction with FCR Controlcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewards
  2. 2026FSPO: Policy-Consistent Risk and Pareto-Feasible Control for Budgeted LLM RL Post-Training
  3. 2026MemRiskBench: Trace-Aware Risk-Preserving Evaluation for Long-Horizon LLM Agents
  4. 2026Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment
  5. 2025Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models
  6. 2025Breaking the Safety-Capability Tradeoff: Reinforcement Learning with Verifiable Rewards Maintains Safety Guardrails in LLMs
  7. 2025Verification-Guided Falsification for Safe RL via Explainable Abstraction and Risk-Aware Exploration
  8. 2026Operational Reframing and Approval-Framed Delegation in Multi-Agent LLM Safety