# Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrieval

arXiv: https://arxiv.org/abs/2204.11373 · HTML: https://mlsanity.com/paper/2204.11373/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 24 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Reading Wikipedia to Answer Open-Domain Questions](https://mlsanity.com/paper/1704.00051/) (2017)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://mlsanity.com/paper/2104.06967/) (2021)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets](https://mlsanity.com/paper/2008.02637/) (2020)
- [End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems](https://mlsanity.com/paper/2010.06028/) (2020)
- [ABNIRML: Analyzing the Behavior of Neural IR Models](https://mlsanity.com/paper/2011.00696/) (2020)
- [Improving Unsupervised Question Answering via Summarization-Informed Question Generation](https://mlsanity.com/paper/2109.07954/) (2021)
- [Are Neural Ranking Models Robust?](https://mlsanity.com/paper/2108.05018/) (2021)
- [Towards Robust Neural Retrieval Models with Synthetic Pre-Training](https://mlsanity.com/paper/2104.07800/) (2021)
- [Synthetic Target Domain Supervision for Open Retrieval QA](https://mlsanity.com/paper/2204.09248/) (2022)
- [Generating Self-Contained and Summary-Centric Question Answer Pairs via Differentiable Reward Imitation Learning](https://mlsanity.com/paper/2109.04689/) (2021)

## Built on by (most cited first, 5 of 5)

- [ProKnow: Process Knowledge for Safety Constrained and Explainable Question Generation for Mental Health Diagnostic Assistance](https://mlsanity.com/paper/2305.08010/) (2023)
- [PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development](https://mlsanity.com/paper/2301.09715/) (2023)
- [Ask To The Point: Open-Domain Entity-Centric Question Generation](https://mlsanity.com/paper/2310.14126/) (2023)
- [Towards Better Generalization in Open-Domain Question Answering by Mitigating Context Memorization](https://mlsanity.com/paper/2404.01652/) (2024)
- [STRIVE: A Think & Improve Approach with Iterative Refinement for Enhancing Question Quality Estimation](https://mlsanity.com/paper/2504.05693/) (2025)

## Abstract (arXiv)

We show that supervised neural information retrieval (IR) models are prone to learning sparse attention patterns over passage tokens, which can result in key phrases including named entities receiving low attention weights, eventually leading to model under-performance. Using a novel targeted synthetic data generation method that identifies poorly attended entities and conditions the generation episodes on those, we teach neural IR to attend more uniformly and robustly to all entities in a given passage. On two public IR benchmarks, we empirically show that the proposed method helps improve both the model's attention patterns and retrieval performance, including in zero-shot settings.