mlsanityData to 7 Oct 2026

Not All Negative Samples Are Equal: LLMs Learn Better from Plausible Reasoning

arXiv 2602.03516 ↗First posted 3 Feb 2026cs.LG cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Learning from negative samples holds great promise for improving Large Language Model (LLM) reasoning capability, yet existing methods treat all incorrect responses as equally informative, overlooking the crucial role of sample quality. To address this, we propose Plausible Negative Samples (PNS), a method that synthesizes high-quality negative samples exhibiting expected format and structural coherence while ultimately yielding incorrect answers. PNS trains a dedicated model via reverse reinforcement learning (RL) guided by a composite reward combining format compliance, accuracy inversion, reward model assessment, and chain-of-thought evaluation, generating responses nearly indistinguishable from correct solutions. We further validate PNS as a plug-and-play data source for preference optimization across three backbone models on seven mathematical reasoning benchmarks. Results demonstrate that PNS consistently outperforms other negative sample synthesis methods, achieving an average improvement of 2.03% over RL-trained models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 46 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

21 of 46 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun
  2. 2025A Minimalist Approach to LLM Reasoning: from Rejection Sampling to Reinforce15 Apr
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025Process Reinforcement through Implicit Rewards3 Feb
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMs22 Jan
  8. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervision5 Jun

Other references · 13

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2024OpenAI o1 System Cardcited by 1,220
  7. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  8. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  9. 2024Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agentscited by 49
  10. 2025Kardia-R1: Unleashing LLMs to Reason toward Understanding and Empathy for Emotional Support via Rubric-as-Judge Reinforcement Learningcited by 22
  11. 2025How much do LLMs learn from negative examples?cited by 6
  12. 2026Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalizationcited by 5
1 more
  1. 2025STARS: Synchronous Token Alignment for Robust Supervision in Large Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning
  2. 2025Unearthing Gems from Stones: Policy Optimization with Negative Sample Augmentation for LLM Reasoning
  3. 2025Hard Negative Sample-Augmented DPO Post-Training for Small Language Models
  4. 2026Adaptive Negative Reinforcement for LLM Reasoning:Dynamically Balancing Correction and Diversity in RLVR
  5. 2026ResRL: Boosting LLM Reasoning via Negative Sample Projection Residual Reinforcement Learning
  6. 2025TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoning
  7. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs
  8. 2025Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning