Agentic Meta-Orchestrator for Multi-task Copilots

arXiv 2510.22781 ↗First posted 26 Oct 2025cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Microsoft Copilot suites serve as the universal entry point for various agents skilled in handling important tasks, ranging from assisting a customer with product purchases to detecting vulnerabilities in corporate programming code. Each agent can be powered by language models, software engineering operations, such as database retrieval, and internal & external knowledge. The repertoire of a copilot can expand dynamically with new agents. This requires a robust orchestrator that can distribute tasks from user prompts to the right agents. In this work, we propose an Agentic Meta-orchestrator (AMO) for handling multiple tasks and scalable agents in copilot services, which can provide both natural language and action responses. We will also demonstrate the planning that leverages meta-learning, i.e., a trained decision tree model for deciding the best inference strategy among various agents/models. We showcase the effectiveness of our AMO through two production use cases: Microsoft 365 (M365) E-Commerce Copilot and code compliance copilot. M365 E-Commerce Copilot advertises Microsoft products to external customers to promote sales success. The M365 E-Commerce Copilot provides up-to-date product information and connects to multiple agents, such as relational databases and human customer support. The code compliance copilot scans the internal DevOps code to detect known and new compliance issues in pull requests (PR).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 47 of its 68 references that are in the map. First posted on arXiv 26 Oct 2025.

Builds on

47 of 68 references.

Closest ancestors · 6

Same direction, newest first

Other references · 41

2017Attention Is All You Needcited by 60,361
2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
2024The Llama 3 Herd of Modelscited by 14,715
2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211

29 more

2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
2022Self-critiquing models for assisting human evaluatorscited by 380
2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 350
2023Large Language Models are Effective Text Rankers with Pairwise Ranking Promptingcited by 307
2024RAFT: Adapting Language Model to Domain Specific RAGcited by 262
2019BERT for Joint Intent Classification and Slot Fillingcited by 196
2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learningcited by 192
2021Meta-learning via Language Model In-context Tuningcited by 164
2022MulT: An End-to-End Multitask Learning Transformercited by 48
2018Weakly-Supervised Hierarchical Text Classificationcited by 42
2022A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methodscited by 42
2022Meta Learning for Natural Language Processing: A Surveycited by 26
2024TELEClass: Taxonomy Enrichment and LLM-Enhanced Hierarchical Text Classification with Minimal Supervisioncited by 25
2024Self-MoE: Towards Compositional Large Language Models with Self-Specialized Expertscited by 23
2024AutoGen Studio: A No-Code Developer Tool for Building and Debugging Multi-Agent Systemscited by 20
2020Hierarchical Entity Typing via Multi-level Learning to Rankcited by 18
2021A Survey of Natural Language Generationcited by 13
2023ChatGPT in the Age of Generative AI and Large Language Models: A Concise Surveycited by 10
2023Neural Amortized Inference for Nested Multi-agent Reasoningcited by 8
2022RankDNN: Learning to Rank for Few-shot Learningcited by 7
2023Advancements in Federated Learning: Models, Methods, and Privacycited by 7
2020Listwise Learning to Rank by Exploring Unique Ratingscited by 5
2024Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval Modelcited by 4
2023Explicit and Implicit Semantic Ranking Frameworkcited by 3
2023Reinforcement Learning Fine-tuning of Language Models is Biased Towards More Extractable Featurescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way