mlsanityData to 14 Oct 2026

History-Aware Cross-Attention Reinforcement: Self-Supervised Multi Turn and Chain-of-Thought Fine-Tuning with vLLM

arXiv 2506.11108 ↗First posted 8 Jun 2025cs.CLcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

We present CAGSR-vLLM-MTC, an extension of our Self-Supervised Cross-Attention-Guided Reinforcement (CAGSR) framework, now implemented on the high-performance vLLM runtime, to address both multi-turn dialogue and chain-of-thought reasoning. Building upon our original single-turn approach, we first instrumented vLLM's C++/CUDA kernels to asynchronously capture per-layer, per-head cross-attention weights during generation. We then generalized our self-supervised reward function to accumulate attention signals over entire conversation histories and intermediate chain-of-thought steps. We discuss practical trade-offs, including an entropy-based clamping mechanism to prevent attention collapse on early context, and outline future directions for multi-party dialogues and hierarchical reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 14 of its 22 references that are in the map. First posted on arXiv 8 Jun 2025.

Builds on

14 of 22 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2020Learning to summarize from human feedbackcited by 2,588
  8. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  9. 2019Attention is not Explanationcited by 929
  10. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  11. 2019Is Attention Interpretable?cited by 389
  12. 2016How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generationcited by 356
2 more
  1. 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
  2. 2025A Self-Supervised Reinforcement Learning Approach for Fine-Tuning Large Language Models Using Cross-Attention Signalscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
  2. 2025Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning
  3. 2026Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialogue
  4. 2025High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement Learning
  5. 2026AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
  6. 2026MemBuilder: Reinforcing LLMs for Long-Term Memory Construction via Attributed Dense Rewards
  7. 2025ThinkDial: An Open Recipe for Controlling Reasoning Effort in Large Language Models
  8. 2025Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models