mlsanityData to 7 Oct 2026

Detecting RLVR Training Data via Structural Convergence of Reasoning

arXiv 2602.11792 ↗First posted 12 Feb 2026cs.AI cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) is central to training modern reasoning models, but the undisclosed training data raises concerns about benchmark contamination. Unlike pretraining methods, which optimize models using token-level probabilities, RLVR fine-tunes models based on reward feedback from self-generated reasoning trajectories, making conventional likelihood-based detection methods less effective. We show that RLVR induces a distinctive behavioral signature: prompts encountered during RLVR training result in more rigid and similar generations, while unseen prompts retain greater diversity. We introduce Min-kNN Distance, a simple black-box detector that quantifies this collapse by sampling multiple completions for a given prompt and computing the average of the k smallest nearest-neighbor edit distances. Min-kNN Distance requires no access to the reference model or token probabilities. Experiments across multiple RLVR-trained reasoning models show that Min-kNN Distance reliably distinguishes RL-seen examples from unseen ones and outperforms existing membership inference and RL contamination detection baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 31 of its 40 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

31 of 40 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?18 Apr
  2. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs3 Mar
  6. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  7. 2025Process Reinforcement through Implicit Rewards3 Feb
  8. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  9. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations14 Dec
  10. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 21

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2022Red Teaming Language Models with Language Modelscited by 983
  6. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  7. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
  8. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  9. 2024FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AIcited by 227
  10. 2024Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Modelscited by 102
  11. 2023Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibrationcited by 90
  12. 2025Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiadcited by 72
9 more
  1. 2024ReCaLL: Membership Inference via Relative Conditional Log-Likelihoodscited by 61
  2. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 57
  3. 2025JustRL: Scaling a 1.5B LLM with a Simple RL Recipecited by 57
  4. 2025Seed-Coder: Let the Code Model Curate Data for Itselfcited by 46
  5. 2024Creativity Has Left the Chat: The Price of Debiasing Language Modelscited by 29
  6. 2025Leveraging Online Olympiad-Level Math Problems for LLMs Training and Contamination-Resistant Evaluationcited by 29
  7. 2022Rethinking and Refining the Distinct Metriccited by 27
  8. 2025Diverse Preference Learning for Capabilities and Alignmentcited by 25
  9. 2025Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
  2. 2026On the Implicit Reward Overfitting and the Low-rank Dynamics in RLVR
  3. 2026On the Direction of RLVR Updates for LLM Reasoning: Identification and Exploitation
  4. 2026Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots
  5. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  6. 2025The Path Not Taken: RLVR Provably Learns Off the Principals
  7. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  8. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation