mlsanityData to 14 Oct 2026

Learning Selective LLM Autonomy from Copilot Feedback in Enterprise Customer Support Workflows

arXiv 2604.23855 ↗First posted 26 Apr 2026cs.CL cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a deployed system that automates end-to-end customer support workflows inside an enterprise Business Process Management (BPM) platform. The approach is scalable in production and reaches selective automation within two weeks for a new process, leveraging supervision already generated at scale: structured per-case UI interaction traces and low-overhead copilot feedback, where operators either accept a suggestion or provide a correction. A staged deployment pipeline trains a next UI action policy, learns a critic from copilot feedback to calibrate abstention, and executes only high-confidence steps in the background while deferring uncertain decisions to operators and resuming from the updated UI state. This setup lets one operator supervise multiple concurrent sessions and be interrupted only when the system is uncertain. The system operates on a schema-driven view of the BPM interface and includes monitoring and safe fallbacks for production. In production, it automated 45% of sessions and reduced average handling time by 39% without degrading support quality level.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 26 Apr 2026.

Builds on

22 of 32 references.

References · 22

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  7. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 382
  8. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
  9. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  10. 2024OmniParser for Pure Vision Based GUI Agentcited by 198
  11. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agentcited by 130
  12. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 118
10 more
  1. 2021Machine Learning with a Reject Option: A surveycited by 109
  2. 2017Deeply AggreVaTeD: Differentiable Imitation Learning for Sequential Predictioncited by 93
  3. 2024Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignmentcited by 46
  4. 2021TWEETSUMM -- A Dialog Summarization Dataset for Customer Servicecited by 34
  5. 2020TicketTalk: Toward human-level performance with end-to-end, transaction-based dialog systemscited by 18
  6. 2024Attention-driven GUI Grounding: Leveraging Pretrained Multimodal Large Language Models without Fine-Tuningcited by 14
  7. 2025LLM-Friendly Knowledge Representation for Customer Supportcited by 12
  8. 2023Bringing the State-of-the-Art to Customers: A Neural Agent Assistant Framework for Customer Service Supportcited by 6
  9. 2023NatCS: Eliciting Natural Customer Support Dialoguescited by 5
  10. 2025T-pro 2.0: An Efficient Russian Hybrid-Reasoning Model and Playgroundcited by 2

Built on by

Not cited yet.