# Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training

arXiv: https://arxiv.org/abs/2311.09198 · HTML: https://mlsanity.com/paper/2311.09198/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 55 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions](https://mlsanity.com/paper/2212.10509/) (2022)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Compressive Transformers for Long-Range Sequence Modelling](https://mlsanity.com/paper/1911.05507/) (2019)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [Retrieval Augmentation Reduces Hallucination in Conversation](https://mlsanity.com/paper/2104.07567/) (2021)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [Trusting Your Evidence: Hallucinate Less with Context-aware Decoding](https://mlsanity.com/paper/2305.14739/) (2023)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://mlsanity.com/paper/2209.10063/) (2022)
- [ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory](https://mlsanity.com/paper/2306.03901/) (2023)
- [End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering](https://mlsanity.com/paper/2106.05346/) (2021)

## Built on by (most cited first, 30 of 32)

- [Identifying and Mitigating Position Bias of Multi-image Vision-Language Models](https://mlsanity.com/paper/2503.13792/) (2025)
- [Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generation](https://mlsanity.com/paper/2409.11598/) (2024)
- [Shifting Long-Context LLMs Research from Input to Output](https://mlsanity.com/paper/2503.04723/) (2025)
- [Do RAG Systems Really Suffer From Positional Bias?](https://mlsanity.com/paper/2505.15561/) (2025)
- [The Imperative of Conversation Analysis in the Era of LLMs: A Survey of Tasks, Techniques, and Trends](https://mlsanity.com/paper/2409.14195/) (2024)
- [LADM: Long-context Training Data Selection with Attention-based Dependency Measurement for LLMs](https://mlsanity.com/paper/2503.02502/) (2025)
- [Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration](https://mlsanity.com/paper/2505.20625/) (2025)
- [Unstructured Evidence Attribution for Long Context Query Focused Summarization](https://mlsanity.com/paper/2502.14409/) (2025)
- [Hearing the Order: Investigating Position Bias in Large Audio-Language Models](https://mlsanity.com/paper/2510.00628/) (2025)
- [Intelligence Degradation in Long-Context LLMs: Critical Threshold Determination via Natural Length Distribution Analysis](https://mlsanity.com/paper/2601.15300/) (2026)
- [Mitigate Position Bias in Large Language Models via Scaling a Single Dimension](https://mlsanity.com/paper/2406.02536/) (2024)
- [Lost-in-the-Middle in Long-Text Generation: Synthetic Dataset, Evaluation Framework, and Mitigation](https://mlsanity.com/paper/2503.06868/) (2025)
- [Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization](https://mlsanity.com/paper/2508.13993/) (2025)
- [SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine](https://mlsanity.com/paper/2410.17021/) (2024)
- [Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models](https://mlsanity.com/paper/2412.07171/) (2024)
- [SEAL: Scaling to Emphasize Attention for Long-Context Retrieval](https://mlsanity.com/paper/2501.15225/) (2025)
- [RefuteBench 2.0 -- Agentic Benchmark for Dynamic Evaluation of LLM Responses to Refutation Instruction](https://mlsanity.com/paper/2502.18308/) (2025)
- [DoubleDipper: Improving Long-Context LLMs via Context Recycling](https://mlsanity.com/paper/2406.13632/) (2024)
- [END: Early Noise Dropping for Efficient and Effective Context Denoising](https://mlsanity.com/paper/2502.18915/) (2025)
- [Layer-Specific Scaling of Positional Encodings for Superior Long-Context Modeling](https://mlsanity.com/paper/2503.04355/) (2025)
- [Binary Autoencoder for Mechanistic Interpretability of Large Language Models](https://mlsanity.com/paper/2509.20997/) (2025)
- [Uncovering the Role of Initial Saliency in U-Shaped Attention Bias: Scaling Initial Token Weight for Enhanced Long-Text Processing](https://mlsanity.com/paper/2512.13109/) (2025)
- [Abstract2Appendix: Academic Reviews Enhance LLM Long-Context Capabilities](https://mlsanity.com/paper/2411.05232/) (2024)
- [Rank It, Then Ask It: Input Reranking for Maximizing the Performance of LLMs on Symmetric Tasks](https://mlsanity.com/paper/2412.00546/) (2024)
- [Parallel Key-Value Cache Fusion for Position Invariant RAG](https://mlsanity.com/paper/2501.07523/) (2025)
- [Business as Rulesual: A Benchmark and Framework for Business Rule Flow Modeling with LLMs](https://mlsanity.com/paper/2505.18542/) (2025)
- [Towards Position-Robust Talent Recommendation via Large Language Models](https://mlsanity.com/paper/2604.02200/) (2026)
- [Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Models](https://mlsanity.com/paper/2605.28828/) (2026)
- [Improving Answer Extraction in Context-based Question Answering Systems Using LLMs](https://mlsanity.com/paper/2606.06197/) (2026)
- [Unified KV Pooling to Accelerate Long-Context LLM Serving](https://mlsanity.com/paper/2606.14779/) (2026)

## Abstract (arXiv)

While large language models (LLMs) are equipped with longer text input capabilities than before, they are struggling to seek correct information in long contexts. The "lost in the middle" problem challenges most LLMs, referring to the dramatic decline in accuracy when correct information is located in the middle. To overcome this crucial issue, this paper proposes to enhance the information searching and reflection ability of LLMs in long contexts via specially designed tasks called Attention Strengthening Multi-doc QA (ASM QA). Following these tasks, our model excels in focusing more precisely on the desired information. Experimental results show substantial improvement in Multi-doc QA and other benchmarks, superior to state-of-the-art models by 13.7% absolute gain in shuffled settings, by 21.5% in passage retrieval task. We release our model, Ziya-Reader to promote related research in the community.