mlsanityData to 7 Oct 2026

Large Language Models Can Self-Improve in Long-context Reasoning

arXiv 2411.08147 ↗First posted 12 Nov 2024cs.CL cs.AIcited by 29

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have achieved substantial progress in processing long contexts but still struggle with long-context reasoning. Existing approaches typically involve fine-tuning LLMs with synthetic data, which depends on annotations from human experts or advanced models like GPT-4, thus restricting further advancements. To address this issue, we investigate the potential for LLMs to self-improve in long-context reasoning and propose ours, an approach specifically designed for this purpose. This approach is straightforward: we sample multiple outputs for each question, score them with Minimum Bayes Risk, and then apply supervised fine-tuning or preference optimization based on these outputs. Extensive experiments on several leading LLMs demonstrate the effectiveness of ours, with an absolute improvement of 4.2 points for Llama-3.1-8B-Instruct. Furthermore, ours achieves superior performance compared to prior approaches that depend on data produced by human experts or advanced models. We anticipate that this work will open new avenues for self-improvement techniques in long-context scenarios, which are essential for the continual advancement of LLMs.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 48 of its 110 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

48 of 110 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024ALR2: A Retrieve-then-Reason Framework for Long-context Question Answering4 Oct
  2. 2024Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries19 Sep
  3. 2024NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densities16 Jul
  4. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools18 Jun

Other references · 44

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2024Qwen2 Technical Reportcited by 2,076
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  11. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  12. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
32 more
  1. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  2. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  3. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  4. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 390
  5. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  6. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  7. 2023DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Modelscited by 239
  8. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  9. 2023LongNet: Scaling Transformers to 1,000,000,000 Tokenscited by 201
  10. 2024Data Engineering for Scaling Language Models to 128K Contextcited by 183
  11. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  12. 2024How to Train Long-Context Language Models (Effectively)cited by 141
  13. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  14. 2024You Only Cache Once: Decoder-Decoder Architectures for Language Modelscited by 135
  15. 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
  16. 2024Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QAcited by 121
  17. 2024HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughlycited by 119
  18. 2024Make Your LLM Fully Utilize the Contextcited by 118
  19. 2024LongAlign: A Recipe for Long Context Alignment of Large Language Modelscited by 102
  20. 2024Long-Context Language Modeling with Parallel Context Encodingcited by 84
  21. 2024One Thousand and One Pairs: A "novel" challenge for long-context language modelscited by 80
  22. 2024Why Does the Effective Context Length of LLMs Fall Short?cited by 60
  23. 2024Self-Consistency Preference Optimizationcited by 50
  24. 2024LongReward: Improving Long-context Large Language Models with AI Feedbackcited by 34
  25. 2024SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responsescited by 24
  26. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  27. 2024Training Language Models to Critique With Multi-agent Feedbackcited by 16
  28. 2024Extending Llama-3's Context Ten-Fold Overnightcited by 15
  29. 2024What are the Essential Factors in Crafting Effective Long Context Multi-Hop Instruction Datasets? Insights and Best Practicescited by 14
  30. 2024CriticEval: Evaluating Large Language Model as Criticcited by 13
  31. 2024Don't Throw Away Data: Better Sequence Knowledge Distillationcited by 9
  32. 2024Differential Transformercited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 9.

29 papers.

Sort:

By area:

  1. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivizationcited by 124
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  3. 2025SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoningcited by 41
  4. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  5. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  6. 2025LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contextscited by 25
  7. 2025Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervisioncited by 20
  8. 2025LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Datacited by 16
  9. 2025SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularizationcited by 11
  10. 2025Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMscited by 9
  11. 2025Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Modelscited by 8
  12. 2025Position Bias Mitigates Position Bias:Mitigate Position Bias Through Inter-Position Knowledge Distillationcited by 8
17 more
  1. 2025Self-Taught Agentic Long Context Understandingcited by 7
  2. 2025Will Pre-Training Ever End? A First Step Toward Next-Generation Foundation MLLMs via Self-Improving Systematic Cognitioncited by 7
  3. 2025Attention Reveals More Than Tokens: Training-Free Long-Context Reasoning with Attention-guided Retrievalcited by 5
  4. 2025Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Checkcited by 5
  5. 2025Revisiting Long-context Modeling from Context Denoising Perspectivecited by 5
  6. 2025CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capabilitycited by 4
  7. 2025Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimizationcited by 3
  8. 2025Flora: Effortless Context Construction to Arbitrary Length and Scalecited by 2
  9. 2026Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shapingcited by 1
  10. 2026Probing How Scalable Table Data Enhances General Long-Context Reasoningcited by 1
  11. 2026A Decomposition Perspective to Long-context Reasoning for LLMscited by 1
  12. 2026Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoningcited by 1
  13. 2026LongTraceRL: Learning Long-Context Reasoning from Search Agent Trajectories with Rubric Rewardscited by 1
  14. 2026OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modelingcited by 1
  15. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learningcited by 0
  16. 2026Unsupervised Post-Training of Foundation Models: A Surveycited by 0
  17. 2026StateTree: Enhancing Long-Term Dialogue Reasoning via Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LongReason: A Synthetic Long-Context Reasoning Benchmark via Context Expansion
  2. 2022Large Language Models Can Self-Improve
  3. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  4. 2026Reasoning Shift: How Context Silently Shortens LLM Reasoning
  5. 2026π2: Structure-Originated Reasoning Data Improves Long-Context Reasoning Ability of Large Language Models
  6. 2025Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
  7. 2025Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
  8. 2025Double-Checker: Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning