mlsanityData to 14 Oct 2026

AutoMix: Automatically Mixing Language Models

arXiv 2310.12963 ↗First posted 19 Oct 2023cs.CL cs.AIcited by 125

Not on the map. Why?

Cited by 25 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) are now available from cloud API providers in various sizes and configurations. While this diversity offers a broad spectrum of choices, effectively leveraging the options to optimize computational cost and performance remains challenging. In this work, we present Automix, an approach that strategically routes queries to larger LMs, based on the approximate correctness of outputs from a smaller LM. Central to Automix are two key technical contributions. First, it has a few-shot self-verification mechanism, which estimates the reliability of its own outputs without requiring extensive training. Second, given that self-verification can be noisy, it employs a POMDP based router that can effectively select an appropriately sized model, based on answer confidence. Experiments across five language models and five challenging datasets show that Automix consistently surpasses strong baselines, reducing computational cost by over 50% for comparable performance.

Key facts. Cited by 125 papers in the map (data through 14 Oct 2026). Builds on 45 of its 69 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

45 of 69 references.

References · 45

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  8. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  10. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  11. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  12. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
33 more
  1. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  2. 2018CoQA: A Conversational Question Answering Challengecited by 683
  3. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  4. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  5. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  6. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 353
  7. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  8. 2022Confident Adaptive Language Modelingcited by 310
  9. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 308
  10. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  11. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 287
  12. 2022Generating Sequences by Learning to Self-Correctcited by 277
  13. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  14. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  15. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
  16. 2021QuALITY: Question Answering with Long Input Texts, Yes!cited by 208
  17. 2021How Many Data Points is a Prompt Worth?cited by 198
  18. 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
  19. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 176
  20. 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
  21. 2022DocPrompting: Generating Code by Retrieving the Docscited by 153
  22. 2021ContractNLI: A Dataset for Document-level Natural Language Inference for Contractscited by 148
  23. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  24. 2020MuTual: A Dataset for Multi-Turn Dialogue Reasoningcited by 115
  25. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMscited by 115
  26. 2021The Efficiency Misnomercited by 98
  27. 2023Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modelingcited by 95
  28. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 74
  29. 2023On Optimal Caching and Model Multiplexing for Large Model Inferencecited by 20
  30. 2013Learning Finite-State Controllers for Partially Observable Environmentscited by 19
  31. 2021RomeBERT: Robust Training of Multi-Exit BERTcited by 19
  32. 2023DiPlomat: A Dialogue Dataset for Situated Pragmatic Reasoningcited by 14
  33. 2022FLOWGEN: Fast and slow graph generationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 27; 2026: 87.

125 papers.

Sort:

By area:

  1. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  2. 2025Universal Model Routing for Efficient LLM Inferencecited by 86
  3. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  4. 2024A Unified Approach to Routing and Cascading for LLMscited by 80
  5. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  6. 2024Cost-Effective Online Multi-LLM Selection with Versatile Reward Modelscited by 39
  7. 2025CARROT: A Cost Aware Rate Optimal Routercited by 39
  8. 2024Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selectioncited by 33
  9. 2025Prompt-to-Leaderboardcited by 30
  10. 2025Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systemscited by 20
  11. 2025Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolutioncited by 19
  12. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 19
38 more
  1. 2025Cost-Aware Contrastive Routing for LLMscited by 16
  2. 2025Cost-Saving LLM Cascades with Early Abstentioncited by 14
  3. 2026R2-Router: A New Paradigm for LLM Routing with Reasoningcited by 14
  4. 2024PickLLM: Context-Aware RL-Assisted Large Language Model Routingcited by 13
  5. 2025Efficient Training-Free Online Routing for High-Volume Multi-LLM Servingcited by 13
  6. 2026MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routingcited by 13
  7. 2026When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Modelscited by 13
  8. 2025Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Modelscited by 12
  9. 2025Life-Cycle Routing Vulnerabilities of LLM Routercited by 11
  10. 2025Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routerscited by 11
  11. 2025Cascaded Language Models for Cost-effective Human-AI Decision-Makingcited by 11
  12. 2026SWE-Router: Routing in Multi-turn Agentic Software Engineering Taskscited by 11
  13. 2025Rational Tuning of LLM Cascades via Probabilistic Modelingcited by 10
  14. 2026When Routing Collapses: On the Degenerate Convergence of LLM Routerscited by 10
  15. 2026Agent-as-a-Router: Agentic Model Routing for Coding Taskscited by 10
  16. 2025MESS+: Dynamically Learned Inference-Time LLM Routing in Model Zoos with Service Level Guaranteescited by 9
  17. 2026TRIM: Hybrid Inference via Targeted Stepwise Routing in Multi-Step Reasoning Taskscited by 8
  18. 2026Adaptive Vision-Language Model Routing for Computer Use Agentscited by 8
  19. 2026The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong Worldcited by 8
  20. 2025LLM Routing with Dueling Feedbackcited by 7
  21. 2026Effective LoRA Adapter Routing using Task Representationscited by 6
  22. 2026vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Modelscited by 6
  23. 2026Cross-Model Disagreement as a Label-Free Correctness Signalcited by 6
  24. 2026Uno-Orchestra: Parsimonious Agent Routing via Selective Delegationcited by 6
  25. 2026TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routingcited by 6
  26. 2024Efficiently Deploying LLMs with Controlled Riskcited by 5
  27. 2024Verbosity neq Veracity: Demystify Verbosity Compensation Behavior of Large Language Modelscited by 5
  28. 2025Cascadia: An Efficient Cascade Serving System for Large Language Modelscited by 5
  29. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniquescited by 5
  30. 2025Economic Evaluation of LLMscited by 5
  31. 2025Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Trainingcited by 5
  32. 2025DiSRouter: Distributed Self-Routing for LLM Selectionscited by 5
  33. 2025UncertainSAM: Fast and Efficient Uncertainty Quantification of the Segment Anything Modelcited by 4
  34. 2025ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Modelscited by 4
  35. 2026PROTEUS: SLA-Aware Routing via Lagrangian RL for Multi-LLM Serving Systemscited by 4
  36. 2026Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Modelscited by 4
  37. 2026Is Escalation Worth It? On the Depth of LLM Cascadescited by 4
  38. 2026LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routerscited by 4

50 most cited of 125 All 125 (JSON)