mlsanityData to 7 Oct 2026

AIANO: Enhancing Information Retrieval with AI-Augmented Annotation

arXiv 2602.04579 ↗First posted 4 Feb 2026cs.IR cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

The rise of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) has rapidly increased the need for high-quality, curated information retrieval datasets. These datasets, however, are currently created with off-the-shelf annotation tools that make the annotation process complex and inefficient. To streamline this process, we developed a specialized annotation tool - AIANO. By adopting an AI-augmented annotation workflow that tightly integrates human expertise with LLM assistance, AIANO enables annotators to leverage AI suggestions while retaining full control over annotation decisions. In a within-subject user study (n = 15), participants created question-answering datasets using both a baseline tool and AIANO. AIANO nearly doubled annotation speed compared to the baseline while being easier to use and improving retrieval accuracy. These results demonstrate that AIANO's AI-augmented approach accelerates and enhances dataset creation for information retrieval tasks, advancing annotation capabilities in retrieval-intensive domains.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 14 of its 25 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

14 of 25 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems25 Jun
  2. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems16 Nov
  3. 2023Ragas: Automated Evaluation of Retrieval Augmented Generation26 Sep

Other references · 11

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  5. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  6. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  7. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 93
  8. 2023Automated Annotation with Generative AI Requires Validationcited by 75
  9. 2024OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domaincited by 28
  10. 2025GaRAGe: A Benchmark with Grounding Annotations for RAG Evaluationcited by 18
  11. 2024Evaluating D-MERIT of Partial-annotation on Information Retrievalcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Retrieval-Augmented Generation for AI-Generated Content: A Survey
  2. 2025Efficacy of AI RAG Tools for Complex Information Extraction and Data Annotation Tasks: A Case Study Using Banks Public Disclosures
  3. 2025Utility-Focused LLM Annotation for Retrieval and Retrieval-Augmented Generation
  4. 2024Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study on Pittsburgh and CMU
  5. 2024Leveraging Retrieval-Augmented Generation for Persian University Knowledge Retrieval
  6. 2025A Knowledge Graph and a Tripartite Evaluation Framework Make Retrieval-Augmented Generation Scalable and Transparent
  7. 2024Deploying Large Language Models With Retrieval Augmented Generation
  8. 2025A Survey on Knowledge-Oriented Retrieval-Augmented Generation