mlsanityData to 7 Oct 2026

SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidence

arXiv 2502.08767 ↗First posted 12 Feb 2025cs.CL cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Providing Language Models (LMs) with relevant evidence in the context (either via retrieval or user-provided) can significantly improve their ability to provide better-grounded responses. However, recent studies have found that LMs often struggle to fully comprehend and utilize key evidence from the context, especially when it contains noise and irrelevant information, an issue common in real-world scenarios. To address this, we propose SelfElicit, an inference-time approach that helps LMs focus on key contextual evidence through self-guided explicit highlighting. By leveraging the inherent evidence-finding capabilities of LMs using the attention scores of deeper layers, our method automatically identifies and emphasizes key evidence within the input context, facilitating more accurate and grounded responses without additional training or iterative prompting. We demonstrate that SelfElicit brings consistent and significant improvement on multiple evidence-based QA tasks for various LM families while maintaining computational efficiency. Our code and documentation are available at https://github.com/ZhiningLiu1998/SelfElicit.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 49 of its 94 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

49 of 94 references.

References · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  7. 2024Qwen2 Technical Reportcited by 2,076
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  10. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  11. 2024Large Language Models: A Surveycited by 506
  12. 2016NewsQA: A Machine Comprehension Datasetcited by 330
37 more
  1. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  2. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  3. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  4. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  5. 2024InstructRAG: Instructing Retrieval-Augmented Generation via Self-Synthesized Rationalescited by 100
  6. 2020Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender Systemcited by 94
  7. 2023Attention Sorting Combats Recency Bias In Long Context Language Modelscited by 89
  8. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrationscited by 78
  9. 2024Discrete-state Continuous-time Diffusion for Graph Generationcited by 73
  10. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  11. 2024Enhancing Contextual Understanding in Large Language Models through Contrastive Decodingcited by 49
  12. 2021EE-Net: Exploitation-Exploration Neural Networks in Contextual Banditscited by 43
  13. 2024In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigationcited by 43
  14. 2024How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmarkcited by 39
  15. 2023Class-Imbalanced Graph Learning without Class Rebalancingcited by 29
  16. 2019Self-paced Ensemble for Highly Imbalanced Massive Data Classificationcited by 28
  17. 2023Ensuring User-side Fairness in Dynamic Recommender Systemscited by 27
  18. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  19. 2020MESA: Boost Ensemble Imbalanced Learning with MEta-SAmplercited by 25
  20. 2024Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyondcited by 25
  21. 2024LLM-Forest: Ensemble Learning of LLMs with Graph-Augmented Prompts for Data Imputationcited by 25
  22. 2023Hierarchical Multi-Marginal Optimal Transport for Network Alignmentcited by 24
  23. 2023Adaptive Test-Time Personalization for Federated Learningcited by 23
  24. 2020KompaRe: A Knowledge Graph Comparative Reasoning Systemcited by 20
  25. 2024Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphscited by 20
  26. 2022Comprehensive Fair Meta-learned Recommender Systemcited by 18
  27. 2024PyG-SSL: A Graph Self-Supervised Learning Toolkitcited by 14
  28. 2024Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?cited by 13
  29. 2024Gradient Compressed Sensing: A Query-Efficient Gradient Estimator for High-Dimensional Zeroth-Order Optimizationcited by 12
  30. 2024WAPITI: A Watermark for Finetuned Open-Source LLMscited by 12
  31. 2025TUCKET: A Tensor Time Series Data Structure for Efficient and Accurate Factor Analysis over Time Rangescited by 12
  32. 2024Ask, and it shall be given: On the Turing completeness of promptingcited by 10
  33. 2024AIM: Attributing, Interpreting, Mitigating Data Unfairnesscited by 9
  34. 2024PopALM: Popularity-Aligned Language Models for Social Media Trendy Response Predictioncited by 8
  35. 2021IMBENS: Ensemble Class-imbalanced Learning in Pythoncited by 6
  36. 2023MLP Fusion: Towards Efficient Fine-tuning of Dense and Mixture-of-Experts Language Modelscited by 6
  37. 2025LLM-RecG: A Semantic Bias-Aware Framework for Zero-Shot Sequential Recommendationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 8.

18 papers.

Sort:

By area:

  1. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  2. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  3. 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decouplingcited by 16
  4. 2025Empowering GraphRAG with Knowledge Filtering and Integrationcited by 15
  5. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  6. 2025AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelismcited by 7
  7. 2025LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layerscited by 3
  8. 2025Continual Low-Rank Adapters for LLM-based Generative Recommender Systemscited by 2
  9. 2025Copy-Paste to Mitigate Large Language Model Hallucinationscited by 1
  10. 2025GOLD PANNING: Strategic Context Shuffling for Needle-in-Haystack Reasoningcited by 1
  11. 2026Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMscited by 1
  12. 2026On Theoretically-Driven LLM Agents for Multi-Dimensional Discourse Analysiscited by 1
6 more
  1. 2026Magnifying What Matters: Attention-Guided Adaptive Rendering for Visual Text Comprehensioncited by 1
  2. 2026Attention-guided Evidence Grounding for Spoken Question Answeringcited by 0
  3. 2026Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answeringcited by 0
  4. 2026Learning Evidence Highlighting for Frozen LLMscited by 0
  5. 2026On Improving Faithfulness of Podcasts from Documentscited by 0
  6. 2026Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Modelscited by 0