mlsanityData to 14 Oct 2026

Evaluation and Continual Improvement for an Enterprise AI Assistant

arXiv 2407.12003 ↗First posted 15 Jun 2024cs.HCcited by 7

Not on the map. Why?

Abstract (arXiv)

The development of conversational AI assistants is an iterative process with multiple components. As such, the evaluation and continual improvement of these assistants is a complex and multifaceted problem. This paper introduces the challenges in evaluating and improving a generative AI assistant for enterprises, which is under active development, and how we address these challenges. We also share preliminary results and discuss lessons learned.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Jun 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2024HierTOD: A Task-Oriented Dialogue System Driven by Hierarchical Goalscited by 6
  2. 2026Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluationscited by 3
  3. 2024ELOQ: Resources for Enhancing LLM Detection of Out-of-Scope Questionscited by 2
  4. 2024Enhancing Discoverability in Enterprise Conversational Systems with Proactive Question Suggestionscited by 2
  5. 2025Evaluation and Incident Prevention in an Enterprise AI Assistantcited by 0
  6. 2025Adobe Summit Concierge Evaluation with Human in the Loopcited by 0
  7. 2026Designing Reliable LLM-as-a-Judge Measurement Systems for Multi-Turn Business Agentscited by 0