# SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidence

arXiv: https://arxiv.org/abs/2502.08767 · HTML: https://mlsanity.com/paper/2502.08767/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 94 references that are in the map.
- First posted on arXiv 12 Feb 2025.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [SnapKV: LLM Knows What You are Looking for Before Generation](https://mlsanity.com/paper/2404.14469/) (2024)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Trusting Your Evidence: Hallucinate Less with Context-aware Decoding](https://mlsanity.com/paper/2305.14739/) (2023)
- [The Power of Noise: Redefining Retrieval for RAG Systems](https://mlsanity.com/paper/2401.14887/) (2024)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://mlsanity.com/paper/1910.09753/) (2019)
- [InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationales](https://mlsanity.com/paper/2406.13629/) (2024)
- [Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System](https://mlsanity.com/paper/2010.15363/) (2020)
- [Attention Sorting Combats Recency Bias In Long Context Language Models](https://mlsanity.com/paper/2310.01427/) (2023)
- [Overthinking the Truth: Understanding how Language Models Process False Demonstrations](https://mlsanity.com/paper/2307.09476/) (2023)
- [Discrete-state Continuous-time Diffusion for Graph Generation](https://mlsanity.com/paper/2405.11416/) (2024)
- [Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models](https://mlsanity.com/paper/2309.15098/) (2023)
- [Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding](https://mlsanity.com/paper/2405.02750/) (2024)
- [In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation](https://mlsanity.com/paper/2403.01548/) (2024)
- [EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits](https://mlsanity.com/paper/2110.03177/) (2021)
- [How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmark](https://mlsanity.com/paper/2406.06647/) (2024)
- [Class-Imbalanced Graph Learning without Class Rebalancing](https://mlsanity.com/paper/2308.14181/) (2023)
- [Self-paced Ensemble for Highly Imbalanced Massive Data Classification](https://mlsanity.com/paper/1909.03500/) (2019)
- [Ensuring User-side Fairness in Dynamic Recommender Systems](https://mlsanity.com/paper/2308.15651/) (2023)
- [Heterogeneous Contrastive Learning for Foundation Models and Beyond](https://mlsanity.com/paper/2404.00225/) (2024)

## Built on by (most cited first, 18 of 18)

- [Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models](https://mlsanity.com/paper/2510.04618/) (2025)
- [WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2505.16421/) (2025)
- [HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling](https://mlsanity.com/paper/2510.00054/) (2025)
- [Empowering GraphRAG with Knowledge Filtering and Integration](https://mlsanity.com/paper/2503.13804/) (2025)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism](https://mlsanity.com/paper/2506.03700/) (2025)
- [LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers](https://mlsanity.com/paper/2507.04404/) (2025)
- [Copy-Paste to Mitigate Large Language Model Hallucinations](https://mlsanity.com/paper/2510.00508/) (2025)
- [Continual Low-Rank Adapters for LLM-based Generative Recommender Systems](https://mlsanity.com/paper/2510.25093/) (2025)
- [GOLD PANNING: Strategic Context Shuffling for Needle-in-Haystack Reasoning](https://mlsanity.com/paper/2510.09770/) (2025)
- [Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMs](https://mlsanity.com/paper/2602.07778/) (2026)
- [On Theoretically-Driven LLM Agents for Multi-Dimensional Discourse Analysis](https://mlsanity.com/paper/2602.13713/) (2026)
- [Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2604.01280/) (2026)
- [Magnifying What Matters: Attention-Guided Adaptive Rendering for Visual Text Comprehension](https://mlsanity.com/paper/2606.12898/) (2026)
- [Attention-guided Evidence Grounding for Spoken Question Answering](https://mlsanity.com/paper/2603.16292/) (2026)
- [Learning Evidence Highlighting for Frozen LLMs](https://mlsanity.com/paper/2604.22565/) (2026)
- [On Improving Faithfulness of Podcasts from Documents](https://mlsanity.com/paper/2607.21961/) (2026)
- [Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models](https://mlsanity.com/paper/2609.16795/) (2026)

## Abstract (arXiv)

Providing Language Models (LMs) with relevant evidence in the context (either via retrieval or user-provided) can significantly improve their ability to provide better-grounded responses. However, recent studies have found that LMs often struggle to fully comprehend and utilize key evidence from the context, especially when it contains noise and irrelevant information, an issue common in real-world scenarios. To address this, we propose SelfElicit, an inference-time approach that helps LMs focus on key contextual evidence through self-guided explicit highlighting. By leveraging the inherent evidence-finding capabilities of LMs using the attention scores of deeper layers, our method automatically identifies and emphasizes key evidence within the input context, facilitating more accurate and grounded responses without additional training or iterative prompting. We demonstrate that SelfElicit brings consistent and significant improvement on multiple evidence-based QA tasks for various LM families while maintaining computational efficiency. Our code and documentation are available at https://github.com/ZhiningLiu1998/SelfElicit.