Can Language Models Critique Themselves? Investigating Self-Feedback for Retrieval Augmented Generation at BioASQ 2025
On the map: Retrieval-augmented generation → Agentic RAG, agents and memory
also in 1 more
Abstract (arXiv)
Agentic Retrieval Augmented Generation (RAG) and 'deep research' systems aim to enable autonomous search processes where Large Language Models (LLMs) iteratively refine outputs. However, applying these systems to domain-specific professional search, such as biomedical research, presents challenges, as automated systems may reduce user involvement and misalign with expert information needs. Professional search tasks often demand high levels of user expertise and transparency. The BioASQ CLEF 2025 challenge, using expert-formulated questions, can serve as a platform to study these issues. We explored the performance of current reasoning and nonreasoning LLMs like Gemini-Flash 2.0, o3-mini, o4-mini and DeepSeek-R1. A key aspect of our methodology was a self-feedback mechanism where LLMs generated, evaluated, and then refined their outputs for query expansion and for multiple answer types (yes/no, factoid, list, ideal). We investigated whether this iterative self-correction improves performance and if reasoning models are more capable of generating useful feedback. Preliminary results indicate varied performance for the self-feedback strategy across models and tasks. This work offers insights into LLM self-correction and informs future work on comparing the effectiveness of LLM-generated feedback with direct human expert input in these search systems.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 14 of its 24 references that are in the map. First posted on arXiv 7 Aug 2025.
Builds on
14 of 24 references.
References · 14
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
- 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
- 2023Evaluating Verifiability in Generative Search Enginescited by 297
- 2024The Use of Generative Search Engines for Knowledge Work and Complex Taskscited by 18
- 2024Can Open-Source LLMs Compete with Commercial Models? Exploring the Few-Shot Performance of Current GPT Models in Biomedical Taskscited by 6
- 2021Interactive query expansion for professional search applicationscited by 5
- 2023Is ChatGPT a Biomedical Expert? -- Exploring the Zero-Shot Performance of Current GPT Models in Biomedical Taskscited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025RAGCap-Bench: Benchmarking Capabilities of LLMs in Agentic Retrieval Augmented Generation Systems
- 2024RA-ISF: Learning to Answer and Understand from Retrieval Augmentation via Iterative Self-Feedback
- 2026Is Agentic RAG worth it? An experimental comparison of RAG approaches
- 2024Auto-RAG: Autonomous Retrieval-Augmented Generation for Large Language Models
- 2025InfoDeepSeek: Benchmarking Agentic Information Seeking for Retrieval-Augmented Generation
- 2025Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG
- 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces
- 2026Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wiki