mlsanityData to 7 Oct 2026

How to Train Your Advisor: Steering Black-Box LLMs with Advisor Models

arXiv 2510.02453 ↗First posted 2 Oct 2025cs.LG cs.AI cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Frontier language models are deployed as black-box services, where model weights cannot be modified and customization is limited to prompting. We introduce Advisor Models, a method to train small open-weight models to generate dynamic, per-instance natural language advice that improves the capabilities of black-box frontier models. Advisor Models improve GPT-5.2's performance on RuleArena (Taxes) by 27.4%, reduce Gemini 3 Pro's steps taken in SWE agent tasks by 24.6%, and outperform static prompt optimizers in personalizing GPT-5 to user preferences (85-100% vs. 40-60%). We also find that advisors are transferable: an advisor trained with a low-cost student model still transfers improvements to a frontier model. Moreover, Advisor Models are robust: we observe no degradation on other benchmarks than the pipeline is trained on. Our method shows how to perform parametric optimization for black-box frontier models in a practical and cost-effective way.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 32 of its 46 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

32 of 46 references.

References · 32

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  11. 2023Large Language Models as Optimizerscited by 830
  12. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
20 more
  1. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  2. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  3. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  4. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  5. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  6. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  7. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 168
  8. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 150
  9. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
  10. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
  11. 2023Guiding Large Language Models via Directional Stimulus Promptingcited by 94
  12. 2023A Benchmark for Learning to Translate a New Language from One Grammar Bookcited by 74
  13. 2024Iterative Data Smoothing: Mitigating Reward Overfitting and Overoptimization in RLHFcited by 57
  14. 2022IDPG: An Instance-Dependent Prompt Generation Methodcited by 43
  15. 2024PRewrite: Prompt Rewriting with Reinforcement Learningcited by 43
  16. 2023Integrating Summarization and Retrieval for Enhanced Personalization via Large Language Modelscited by 40
  17. 2024RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarioscited by 26
  18. 2023Balancing Specialized and General Skills in LLMs: The Impact of Modern Tuning and Data Strategycited by 18
  19. 2025FSPO: Few-Shot Optimization of Synthetic Preferences Personalizes to Real Userscited by 14
  20. 2024Matryoshka Pilot: Learning to Drive Black-Box LLMs with LLMscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 18.

20 papers.

Sort:

By area:

  1. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  2. 2026Remember When It Matters: Proactive Memory Agent for Long-Horizon Agentscited by 10
  3. 2026Shepherd: Enabling Programmable Meta-Agents via Reversible Agentic Execution Tracescited by 8
  4. 2025Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learningcited by 7
  5. 2026PACEvolve++: Improving Test-time Learning for Evolutionary Search Agentscited by 3
  6. 2026Steer, Don't Solve: Training Small Critic Models for Large Code Agentscited by 3
  7. 2026OpenJarvis: Personal AI, On Personal Devicescited by 2
  8. 2026Matryoshka Agent: Unfolding Sub-Agents for Long-Horizon Machine Learning Engineeringcited by 2
  9. 2026From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memorycited by 1
  10. 2026CLEAR: Context Augmentation from Contrastive Learning of Experience via Agentic Reflectioncited by 0
  11. 2026ExecTune: Effective Steering of Black-Box LLMs with Guide Modelscited by 0
  12. 2026Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experiencecited by 0
8 more
  1. 2026Recon: Reconstruction-Guided Reasoning Synthesis for User Modelingcited by 0
  2. 2026When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systemscited by 0
  3. 2026The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Taskscited by 0
  4. 2026Agents Can Use Base Models to Evade AI Detectioncited by 0
  5. 2026FORGE: Form-Optimal Routing of Grounded Evidence for Frozen LLM Agentscited by 0
  6. 2026AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillationcited by 0
  7. 2026Turbo Harness: Instance-Adaptive Harness Optimizationcited by 0
  8. 2026Training Advisors for LLM Agents from Task Outcomescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Are Generative AI Agents Effective Personalized Financial Advisors?
  2. 2025EDGE: Efficient Data Selection for LLM Agents via Guideline Effectiveness
  3. 2024Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning
  4. 2026Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents
  5. 2024A Teacher Is Worth A Million Instructions
  6. 2026Frontier Learning: Training LLM Reasoners at the Edge of Capability
  7. 2024Enhancing the Reasoning Capabilities of Small Language Models via Solution Guidance Fine-Tuning
  8. 2026Model Spec Midtraining: Improving How Alignment Training Generalizes