mlsanityData to 14 Oct 2026

WST: Weak-to-Strong Knowledge Transfer via Reinforcement Learning

arXiv 2508.16741 ↗First posted 22 Aug 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Effective prompt engineering remains a challenging task for many applications. We introduce Weak-to-Strong Transfer (WST), an automatic prompt engineering framework where a small "Teacher" model generates instructions that enhance the performance of a much larger "Student" model. Unlike prior work, WST requires only a weak teacher, making it efficient and broadly applicable in settings where large models are closed-source or difficult to fine-tune. Using reinforcement learning, the Teacher Model's instructions are iteratively improved based on the Student Model's outcomes, yielding substantial gains across reasoning (MATH-500, GSM8K) and alignment (HH-RLHF) benchmarks - 98% on MATH-500 and 134% on HH-RLHF - and surpassing baselines such as GPT-4o-mini and Llama-70B. These results demonstrate that small models can reliably scaffold larger ones, unlocking latent capabilities while avoiding misleading prompts that stronger teachers may introduce, establishing WST as a scalable solution for efficient and safe LLM prompt refinement.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 5 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

5 of 5 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025PRL: Prompts from Reinforcement Learning20 May

Other references · 4

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  2. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
  3. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  4. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 175

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RLPE: Teaching LLMs to Prompt with Reinforcement Learning
  2. 2026Prompting Policies for Multi-step Reasoning and Tool-Use in Black-box LLMs with Iterative Distillation of Experience
  3. 2025Prompt-R1: Collaborative Automatic Prompting Framework via End-to-end Reinforcement Learning
  4. 2024PRewrite: Prompt Rewriting with Reinforcement Learning
  5. 2026LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL
  6. 2025Prompt Curriculum Learning for Efficient LLM Post-Training
  7. 2025You Don't Need Prompt Engineering Anymore: The Prompting Inversion
  8. 2026Weak-to-Strong Generalization via Direct On-Policy Distillation