mlsanityData to 14 Oct 2026

Context Filtering with Reward Modeling in Question Answering

arXiv 2412.11707 ↗First posted 16 Dec 2024cs.CLcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

Question Answering (QA) in NLP is the task of finding answers to a query within a relevant context retrieved by a retrieval system. Yet, the mix of relevant and irrelevant information in these contexts can hinder performance enhancements in QA tasks. To address this, we introduce a context filtering approach that removes non-essential details, summarizing crucial content through Reward Modeling. This method emphasizes keeping vital data while omitting the extraneous during summarization model training. We offer a framework for developing efficient QA models by discerning useful information from dataset pairs, bypassing the need for costly human evaluation. Furthermore, we show that our approach can significantly outperform the baseline, as evidenced by a 6.8-fold increase in the EM Per Token (EPT) metric, which we propose as a measure of token efficiency, indicating a notable token-efficiency boost for low-resource settings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 23 of its 32 references that are in the map. First posted on arXiv 16 Dec 2024.

Builds on

23 of 32 references.

References · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2020Longformer: The Long-Document Transformercited by 3,413
  9. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  11. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
11 more
  1. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  2. 2019ELI5: Long Form Question Answeringcited by 575
  3. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 573
  4. 2020Autoregressive Entity Retrievalcited by 334
  5. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  6. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 256
  7. 2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
  8. 2021Improving Factual Consistency of Abstractive Summarization via Question Answeringcited by 55
  9. 2020Revisiting Design Choices in Proximal Policy Optimizationcited by 38
  10. 2021Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehensioncited by 10
  11. 2023Knowledge Corpus Error in Question Answeringcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Less Is More: Elevating RAG via Performance-Driven Context Compressioncited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2023PRCA: Fitting Black-Box Large Language Models for Retrieval Question Answering via Pluggable Reward-Driven Contextual Adapter
  2. 2025Efficient Context Selection for Long-Context QA: No Tuning, No Iteration, Just Adaptive-k
  3. 2026Inferential Question Answering
  4. 2026EASE-TTT: Evidence-Aligned Selective Test-Time Training for Long-Context Question Answering
  5. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference
  6. 2025Semantic Caching of Contextual Summaries for Efficient Question-Answering with Language Models
  7. 2026SAGE: Selective Attention-Guided Extraction for Token-Efficient Document Indexing ↗
  8. 2025Revisiting NLI: Towards Cost-Effective and Human-Aligned Metrics for Evaluating LLMs in Question Answering