mlsanityData to 14 Oct 2026

DE-Venus: A Data-Efficient RLVR Framework for Large Language Models

arXiv 2609.03324 ↗First posted 3 Sep 2026cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning, but its practical scaling is constrained by expensive on-policy rollouts and the cost of obtaining reliable targets at scale. Existing methods address sample selection, incomplete supervision, or noisy labels separately, often entangling supervision logic with distributed training and hindering controlled comparison and reuse. We present DE-Venus, a unified framework for data-efficient RLVR that treats supervision as evolving state across data preparation and policy optimization. It organizes this lifecycle into three modules: Active Data Selection allocates training and annotation budgets; Weak Supervision Construction derives learning signals from unlabeled examples; and Training-Time Supervision Refinement filters or corrects unreliable supervision. DE-Venus supports seven representative methods and a data-selection pipeline by expressing method-specific decisions as offline dataset transitions or online transformations of targets, rewards, batches, and advantages while preserving verl's distributed execution contracts. Across public benchmarks and three business scenarios, separate configurations preserve or improve model quality with only 10% of labels or as little as 13% of relevant data; selected business configurations also reduce observed convergence steps by 63%--75%. DE-Venus thus reduces annotation and training costs without sacrificing scalable RL execution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 38 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

30 of 38 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling3 Jun
  2. 2026Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots3 Jun
  3. 2026Can LLMs Learn to Reason Robustly under Noisy Supervision?5 Apr
  4. 2025TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoning15 Dec
  5. 2025CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs1 Oct
  6. 2025Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models1 Aug
  7. 2025Learning to Reason without External Rewards26 May
  8. 2025VerifyBench: Benchmarking Reference-based Reward Systems for Large Language Models21 May
  9. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning21 May
  10. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  11. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning15 Apr
  12. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning4 Apr
6 more
  1. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  2. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  4. 2025LIMR: Less is More for RL Scaling17 Feb
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  6. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan

Other references · 12

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  8. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  9. 2025Group Sequence Policy Optimizationcited by 835
  10. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certaintycited by 187
  11. 2024Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluationcited by 57
  12. 2025CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Processcited by 51

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement
  2. 2025RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments
  3. 2026Linear Dynamics in the RLVR Training of Large Language Models
  4. 2026Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes
  5. 2026A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
  6. 2025Efficient Reinforcement Learning for Large Language Models with Intrinsic Exploration
  7. 2025Training Language Models to Reason Efficiently
  8. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle