mlsanityData to 7 Oct 2026

PEPS: Quantum-Inspired Reinforcement Learning for Coherent Reasoning Traces in LLMs

arXiv 2509.20105 ↗First posted 24 Sep 2025cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Large Language Models (LLMs) often struggle with maintaining coherent multi-step reasoning traces, particularly in tasks that require a structured logical flow. This work introduces a quantum-inspired approach to address the challenge by incorporating a fidelity-based reward derived from Projected Entangled Pair States (PEPS) into Proximal Policy Optimization. Unlike prior approaches that use direct supervision or contrastive objectives, the proposed method guides learning through structural consistency, offering a novel approach to enforce global coherence in generated reasoning traces. The proposed framework is evaluated using multiple coherence-determining metrics on diverse datasets such as GSM8K, StrategyQA, and EntailmentBank spanning arithmetic, intuitive, and entailment-based reasoning. Results show that the proposed quantum-inspired approach offers significant improvements over supervised, contrastive, and pretrained baseline approaches, highlighting the effectiveness of quantum-inspired fidelity as a foundation to improve reasoning trace coherence in LLMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 31 references that are in the map. First posted on arXiv 24 Sep 2025.

Builds on

28 of 31 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May
  2. 2022Constitutional AI: Harmlessness from AI Feedback15 Dec
  3. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 25

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Learning to summarize from human feedbackcited by 2,562
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  6. 2024A Survey on Large Language Models for Code Generationcited by 646
  7. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  8. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  9. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  10. 2021Explaining Answers with Entailment Treescited by 148
  11. 2023Training Language Models with Language Feedback at Scalecited by 125
  12. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
13 more
  1. 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 110
  2. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 62
  3. 2025LLMs are Greedy Agents: Effects of RL Fine-tuning on Decision-Making Abilitiescited by 36
  4. 2024QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptationcited by 23
  5. 2023Label Supervised LLaMA Finetuningcited by 20
  6. 2024Reading Subtext: Evaluating Large Language Models on Short Story Summarization with Writerscited by 19
  7. 2025C3AI: Crafting and Evaluating Constitutions for Constitutional AIcited by 13
  8. 2024Reinforcement Learning from Human Feedback: Whose Culture, Whose Values, Whose Perspectives?cited by 8
  9. 2024Quantum Large Language Models via Tensor Network Disentanglerscited by 7
  10. 2025Quantum-Enhanced LLM Efficient Fine Tuningcited by 6
  11. 2025From Prompts to Propositions: A Logic-Based Lens on Student-LLM Interactionscited by 4
  12. 2024LLM-Generated Tips Rival Expert-Created Tips in Helping Students Answer Quantum-Computing Questionscited by 3
  13. 2024Quantum-inspired Techniques in Tensor Networks for Industrial Contextscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026QuantumQA: Enhancing Scientific Reasoning via Physics-Consistent Dataset and Verification-Aware Reinforcement Learning
  2. 2026Fine-Tuning Large Language Models for Quantum Reasoning
  3. 2025Quantum Decision Transformers (QDT): Synergistic Entanglement and Interference for Offline Reinforcement Learning
  4. 2026QART: A Quantum-Classical Hybrid Architecture for Long-Horizon Reasoning -- Exploring a Conditional Path toward Quantum Scaling
  5. 2026Fine-Tuning Small Reasoning Models for Quantum Field Theory
  6. 2025Future Policy Approximation for Offline Reinforcement Learning in LLM Reasoning
  7. 2025Local Coherence or Global Validity? Investigating RLVR Traces in Math Domains
  8. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning