mlsanityData to 14 Oct 2026

ReNCE: Learning to Reason by Noise Contrastive Estimation

arXiv 2601.22432 ↗First posted 30 Jan 2026cs.LG cs.CLcited by 1

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

GRPO is a standard approach to endowing pretrained LLMs with reasoning capabilities. It estimates the advantage of an outcome from a group of K outcomes, and promotes those with positive advantages inside a trust region. Since GRPO discriminates between good and bad outcomes softly, it benefits from additional refinements such as asymmetric clipping and zero-variance data filtering. While effective, these refinements require significant empirical insight and can be challenging to identify. We instead propose an explicit contrastive learning approach. Instead of estimating advantages, we bifurcate K outcomes into positive and negative sets, then maximize the likelihood of positive outcomes. Our approach can be viewed as an online instantiation of (multi-label) noise contrastive estimation for LLM reasoning. We validate our method by demonstrating competitive performance on a suite of challenging math benchmarks against strong baselines such as DAPO and online DPO.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 33 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

23 of 33 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025It Takes Two: Your GRPO Is Secretly DPO1 Oct
  2. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts2 Jun
  3. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  4. 2025Trust Region Preference Approximation: A simple and stable reinforcement learning algorithm for LLM reasoning6 Apr
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  7. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 16

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2020Supervised Contrastive Learningcited by 2,829
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  8. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  9. 2025Group Sequence Policy Optimizationcited by 835
  10. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 660
  11. 2019Learning Dense Representations for Entity Retrievalcited by 122
  12. 2024Noise Contrastive Alignment of Language Models with Explicit Rewardscited by 85
4 more
  1. 2021Exploring the Equivalence of Siamese Self-Supervised Learning via A Unified Gradient Frameworkcited by 49
  2. 2021Understanding Hard Negatives in Noise Contrastive Estimationcited by 39
  3. 2021EntQA: Entity Linking as Question Answeringcited by 34
  4. 2025Bridging Offline and Online Reinforcement Learning for LLMscited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Why Semantic Entropy Fails: Geometry-Aware and Calibrated Uncertainty for Policy Optimizationcited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2025Can GRPO Help LLMs Transcend Their Pretraining Origin?
  2. 2025Mitigating Think-Answer Mismatch in LLM Reasoning Through Noise-Aware Advantage Reweighting
  3. 2026Are complicated loss functions necessary for teaching LLMs to reason?
  4. 2025DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning
  5. 2025GRPO-λ: Credit Assignment improves LLM Reasoning
  6. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce
  7. 2026Limits of Difficulty Scaling: Hard Samples Yield Diminishing Returns in GRPO-Tuned SLMs
  8. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening