mlsanityData to 7 Oct 2026

Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question Answering

arXiv 2506.06313 ↗First posted 26 May 2025cs.IR cs.AI cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Existing long-document question answering systems typically process texts as flat sequences or use heuristic chunking, which overlook the discourse structures that naturally guide human comprehension. We present a discourse-aware hierarchical framework that leverages rhetorical structure theory (RST) for long document question answering. Our approach converts discourse trees into sentence-level representations and employs LLM-enhanced node representations to bridge structural and semantic information. The framework involves three key innovations: language-universal discourse parsing for lengthy documents, LLM-based enhancement of discourse relation nodes, and structure-guided hierarchical retrieval. Extensive experiments on four datasets demonstrate consistent improvements over existing approaches through the incorporation of discourse structure, across multiple genres and languages. Moreover, the proposed framework exhibits strong robustness across diverse document types and linguistic settings.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 37 of its 59 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

37 of 59 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2020Longformer: The Long-Document Transformercited by 3,362
  8. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  9. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  12. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
25 more
  1. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  2. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  3. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  4. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  5. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  6. 2018CoQA: A Conversational Question Answering Challengecited by 674
  7. 2017The NarrativeQA Reading Comprehension Challengecited by 590
  8. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  9. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  10. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
  11. 2023Fine-Tuning Language Models with Just Forward Passescited by 337
  12. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  13. 2024mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrievalcited by 233
  14. 2021LongT5: Efficient Text-To-Text Transformer for Long Sequencescited by 232
  15. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 204
  16. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  17. 2022SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrievalcited by 90
  18. 2022Efficient Long-Text Understanding with Short-Text Modelscited by 71
  19. 2020We Need to Talk About Random Splitscited by 70
  20. 2021Dense Hierarchical Retrieval for Open-Domain Question Answeringcited by 33
  21. 2024Grounding Language Model with Chunking-Free In-Context Retrievalcited by 30
  22. 2022How Do We Answer Complex Questions: Discourse Structure of Long-form Answerscited by 19
  23. 2024Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classificationcited by 16
  24. 2023Drilling Down into the Discourse Structure with LLMs for Long Document Question Answeringcited by 8
  25. 2025Cross-Document Cross-Lingual NLI via RST-Enhanced Graph Fusion and Interpretability Predictioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 35
  2. 2025Learning to Extract Rational Evidence via Reinforcement Learning for Retrieval-Augmented Generationcited by 1
  3. 2026AutothinkRAG: Complexity-Aware Control of Retrieval-Augmented Reasoning for Image-Text Interactioncited by 0
  4. 2026Trace Only What You Need: Structure-Aware On-Demand Hypergraph Memory for Long-Document Question Answeringcited by 0
  5. 2026KaLM-Reranker-V1: Fast but Not Late Interaction for Compressed Document Rerankingcited by 0