# RAGAPHENE: A RAG Annotation Platform with Human Enhancements and Edits

arXiv: https://arxiv.org/abs/2508.19272 · HTML: https://mlsanity.com/paper/2508.19272/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Conversational and speech RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 7 of its 16 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Placed in

- Retrieval-augmented generation / [Conversational and speech RAG](https://mlsanity.com/rag/conversational-and-speech-rag/)

## Builds on (most cited first, 7 of 7)

- [FaithDial: A Faithful Benchmark for Information-Seeking Dialogue](https://mlsanity.com/paper/2204.10757/) (2022)
- [MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents](https://mlsanity.com/paper/2109.12595/) (2021)
- [MTRAG: A Multi-Turn Conversational Benchmark for Evaluating Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2501.03468/) (2025)
- [Retrieval Augmented Generation for Domain-specific Question Answering](https://mlsanity.com/paper/2404.14760/) (2024)
- [RAD-Bench: Evaluating Large Language Models Capabilities in Retrieval Augmented Dialogues](https://mlsanity.com/paper/2409.12558/) (2024)
- [Conversational Entity Linking: Problem Definition and Datasets](https://mlsanity.com/paper/2105.04903/) (2021)
- [InspectorRAGet: An Introspection Platform for RAG Evaluation](https://mlsanity.com/paper/2404.17347/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Longitudinal Study on Different Annotator Feedback Loops in Complex RAG Tasks](https://mlsanity.com/paper/2510.11897/) (2025)

## Abstract (arXiv)

Retrieval Augmented Generation (RAG) is an important aspect of conversing with Large Language Models (LLMs) when factually correct information is important. LLMs may provide answers that appear correct, but could contain hallucinated information. Thus, building benchmarks that can evaluate LLMs on multi-turn RAG conversations has become an increasingly important task. Simulating real-world conversations is vital for producing high quality evaluation benchmarks. We present RAGAPHENE, a chat-based annotation platform that enables annotators to simulate real-world conversations for benchmarking and evaluating LLMs. RAGAPHENE has been successfully used by approximately 40 annotators to build thousands of real-world conversations.