# Iterative Hierarchical Attention for Answering Complex Questions over Long Documents

arXiv: https://arxiv.org/abs/2106.00200 · HTML: https://mlsanity.com/paper/2106.00200/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers](https://mlsanity.com/paper/2105.03011/) (2021)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text](https://mlsanity.com/paper/1904.09537/) (2019)
- [Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text](https://mlsanity.com/paper/1809.00782/) (2018)
- [Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval](https://mlsanity.com/paper/2009.12756/) (2020)
- [HIBERT: Document Level Pre-training of Hierarchical Bidirectional Transformers for Document Summarization](https://mlsanity.com/paper/1905.06566/) (2019)
- [Hierarchical Graph Network for Multi-hop Question Answering](https://mlsanity.com/paper/1911.03631/) (2019)
- [Text Segmentation as a Supervised Learning Task](https://mlsanity.com/paper/1803.09337/) (2018)
- [Interpretation of Natural Language Rules in Conversational Machine Reading](https://mlsanity.com/paper/1809.01494/) (2018)
- [Dynamically Fused Graph Network for Multi-hop Reasoning](https://mlsanity.com/paper/1905.06933/) (2019)
- [Extractive Summarization of Long Documents by Combining Global and Local Context](https://mlsanity.com/paper/1909.08089/) (2019)
- [Knowledge Guided Text Retrieval and Reading for Open Domain Question Answering](https://mlsanity.com/paper/1911.03868/) (2019)
- [Answering Open-Domain Questions of Varying Reasoning Steps from Text](https://mlsanity.com/paper/2010.12527/) (2020)
- [Differentiable Reasoning over a Virtual Knowledge Base](https://mlsanity.com/paper/2002.10640/) (2020)
- [Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge](https://mlsanity.com/paper/2007.00849/) (2020)
- [Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading](https://mlsanity.com/paper/2010.01838/) (2020)
- [Multi-granularity hierarchical attention fusion networks for reading comprehension and question answering](https://mlsanity.com/paper/1811.11934/) (2018)
- [A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1609.02745/) (2016)
- [HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions](https://mlsanity.com/paper/2012.15534/) (2020)
- [Reasoning Over Virtual Knowledge Bases With Open Predicate Relations](https://mlsanity.com/paper/2102.07043/) (2021)
- [Language Model Pre-training for Hierarchical Document Representations](https://mlsanity.com/paper/1901.09128/) (2019)

## Built on by (most cited first, 10 of 10)

- [PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents](https://mlsanity.com/paper/2305.14564/) (2023)
- [HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs](https://mlsanity.com/paper/2406.06027/) (2024)
- [Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning](https://mlsanity.com/paper/2311.03734/) (2023)
- [SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models](https://mlsanity.com/paper/2311.09818/) (2023)
- [A Survey on Table-and-Text HybridQA: Concepts, Methods, Challenges and Future Directions](https://mlsanity.com/paper/2212.13465/) (2022)
- [Hybrid Graphs for Table-and-Text based Question Answering using LLMs](https://mlsanity.com/paper/2501.17767/) (2025)
- [TACR: A Table-alignment-based Cell-selection and Reasoning Model for Hybrid Question-Answering](https://mlsanity.com/paper/2305.14682/) (2023)
- [HopPG: Self-Iterative Program Generation for Multi-Hop Question Answering over Heterogeneous Knowledge](https://mlsanity.com/paper/2308.11257/) (2023)
- [Chain of Condition: Construct, Verify and Solve Conditions for Conditional Question Answering](https://mlsanity.com/paper/2408.05442/) (2024)
- [Perturbation-based Active Learning for Question Answering](https://mlsanity.com/paper/2311.02345/) (2023)

## Abstract (arXiv)

We propose a new model, DocHopper, that iteratively attends to different parts of long, hierarchically structured documents to answer complex questions. Similar to multi-hop question-answering (QA) systems, at each step, DocHopper uses a query q to attend to information from a document, combines this ``retrieved'' information with q to produce the next query. However, in contrast to most previous multi-hop QA systems, DocHopper is able to ``retrieve'' either short passages or long sections of the document, thus emulating a multi-step process of ``navigating'' through a long document to answer a question. To enable this novel behavior, DocHopper does not combine document information with q by concatenating text to the text of q, but by combining a compact neural representation of q with a compact neural representation of a hierarchical part of the document, which can potentially be quite large. We experiment with DocHopper on four different QA tasks that require reading long and complex documents to answer multi-hop questions, and show that DocHopper achieves state-of-the-art results on three of the datasets. Additionally, DocHopper is efficient at inference time, being 3--10 times faster than the baselines.