mlsanityData to 14 Oct 2026

Self-evolving Agentic Customer Support System at LinkedIn

arXiv 2608.10224 ↗First posted 10 Aug 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

Enterprise support agents operate in rapidly changing environments where policies, product capabilities, and knowledge bases evolve continuously, making static assistants brittle and costly to maintain. We present LinkedIn's self-evolving agentic support system, which integrates retrieval-augmented generation with evolutionary auto-prompting and a modular, production-aligned evaluation framework to enable safe, continuous improvement without retraining foundation models. The system treats prompts, retrieval, and evaluation as a closed-loop, versioned workflow with operational guardrails. Offline simulations and ablations show clear quality gains over vanilla RAG and baseline agents, including reduced hallucinations and improved response completeness. In a two-week user-randomized A/B test on LinkedIn's production support traffic, the integrated self-evolved workflow increased QA self-serve by 9.0 percentage points, cancellation self-serve by 4.8 points, and routing accuracy by 30.6 points. These results demonstrate a practical path to scalable, self-evolving AI agents in real-world enterprise settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 25 of its 32 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

25 of 32 references.

References · 25

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  8. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  10. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  12. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
13 more
  1. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  2. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  4. 2019Passage Re-ranking with BERTcited by 645
  5. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  6. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  7. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 340
  8. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  9. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 267
  10. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 178
  11. 2020MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answeringcited by 153
  12. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 50
  13. 2025MIRAGE: A Metric-Intensive Benchmark for Retrieval-Augmented Generation Evaluationcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems
  2. 2026ShopEase: A Generative AI-Based Multi-Agent Framework for Intelligent Enterprise Customer Support Using Hybrid Retrieval-Augmented Generation
  3. 2026EvoAgentBench: Benchmarking Agent Self-Evolution via Ability Transfer
  4. 2026Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents
  5. 2025Enabling Self-Improving Agents to Learn at Test Time With Human-In-The-Loop Guidance
  6. 2026AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems
  7. 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent
  8. 2026ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?