# Agent-in-the-Loop: A Data Flywheel for Continuous Improvement in LLM-based Customer Support

arXiv: https://arxiv.org/abs/2510.06674 · HTML: https://mlsanity.com/paper/2510.06674/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 12 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 9 of 9)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [ORPO: Monolithic Preference Optimization without Reference Model](https://mlsanity.com/paper/2403.07691/) (2024)
- [Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2504.12501/) (2025)
- [Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arena](https://mlsanity.com/paper/2407.10627/) (2024)
- [Are LLMs Prescient? A Continuous Evaluation using Daily News as the Oracle](https://mlsanity.com/paper/2411.08324/) (2024)

## Built on by (most cited first, 10 of 10)

- [SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support](https://mlsanity.com/paper/2604.08618/) (2026)
- [Understanding the Challenges in Iterative Generative Optimization with LLMs](https://mlsanity.com/paper/2603.23994/) (2026)
- [SEAR: Schema-Based Evaluation and Routing for LLM Gateways](https://mlsanity.com/paper/2603.26728/) (2026)
- [Iterating Toward Better Search: A Two-Agent Simulation Framework for Evaluating Agentic Search Architectures in E-Commerce](https://mlsanity.com/paper/2606.12924/) (2026)
- [Demand-Driven Context: A Methodology for Building Enterprise Knowledge Bases Through Agent Failure](https://mlsanity.com/paper/2603.14057/) (2026)
- [FRAMES: Guarded and Dual-Objective Skill Evolution for Agents in Policy-Governed Enterprise Workflows](https://mlsanity.com/paper/2608.01772/) (2026)
- [SkillEvo: Self-Renewing Evolution Gradients from Multi-Turn Interaction Feedback](https://mlsanity.com/paper/2608.13120/) (2026)
- [From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale](https://mlsanity.com/paper/2609.05758/) (2026)
- [RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents](https://mlsanity.com/paper/2609.20754/) (2026)
- [State-Grounded Conditioning: Wrapping User-Facing LLM Agents Where Direction Depends on Live State](https://mlsanity.com/paper/2609.27606/) (2026)

## Abstract (arXiv)

We introduce an Agent-in-the-Loop (AITL) framework that implements a continuous data flywheel for iteratively improving an LLM-based customer support system. Unlike standard offline approaches that rely on batch annotations, AITL integrates four key types of annotations directly into live customer operations: (1) pairwise response preferences, (2) agent adoption and rationales, (3) knowledge relevance checks, and (4) identification of missing knowledge. These feedback signals seamlessly feed back into models' updates, reducing retraining cycles from months to weeks. Our production pilot involving US-based customer support agents demonstrated significant improvements in retrieval accuracy (+11.7% recall@75, +14.8% precision@8), generation quality (+8.4% helpfulness) and agent adoption rates (+4.5%). These results underscore the effectiveness of embedding human feedback loops directly into operational workflows to continuously refine LLM-based customer support system.