# Allspark: Weak to Strong Transfer via Alternating Chain of Thought

arXiv: https://arxiv.org/abs/2609.32913 · HTML: https://mlsanity.com/paper/2609.32913/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 26 Sep 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 18 of 18)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems](https://mlsanity.com/paper/2505.11831/) (2025)
- [Kimi K3: Open Frontier Intelligence](https://mlsanity.com/paper/2607.24653/) (2026)
- [Learning to Decode Collaboratively with Multiple Language Models](https://mlsanity.com/paper/2403.03870/) (2024)
- [Weak-to-Strong Reasoning](https://mlsanity.com/paper/2407.13647/) (2024)
- [Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models](https://mlsanity.com/paper/2405.19262/) (2024)
- [APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generation](https://mlsanity.com/paper/2509.18521/) (2025)
- [Weak-to-Strong Generalization via Direct On-Policy Distillation](https://mlsanity.com/paper/2607.05394/) (2026)
- [Breaking the Tokenizer Barrier: On-Policy Distillation across Model Families](https://mlsanity.com/paper/2606.09456/) (2026)
- [Weak-to-Strong On-Policy Distillation](https://mlsanity.com/paper/2607.26246/) (2026)
- [Weak Critics Make Strong Learners: On-Policy Critique Distillation for Scalable Oversight](https://mlsanity.com/paper/2606.00424/) (2026)
- [Score Centering Stabilizes Off-policy Reinforcement Learning](https://mlsanity.com/paper/2609.20807/) (2026)
- [Synergistic Weak-Strong Collaboration by Aligning Preferences](https://mlsanity.com/paper/2504.15188/) (2025)
- [Eliciting Weak-to-Strong Generalization with On-Policy Reverse Distillation](https://mlsanity.com/paper/2609.08798/) (2026)

## Abstract (arXiv)

Recent progress in frontier models has renewed interest in large-scale reinforcement learning (RL), but the cost of generating large-model rollouts makes even testing RL recipes expensive. We ask whether reasoning improvements learned by a small, weak model can benefit a larger, stronger model without using the strong model's rollouts during training. We introduce Allspark, a training and inference framework for weak-to-strong transfer through alternating chains of thought. A weak teacher is trained alongside a frozen copy of the same model; the two alternate reasoning segments, and the frozen model produces the final answer. At inference time, a stronger student replaces the frozen training partner, while both models remain fixed. Because they communicate through text, the teacher can steer students from different model families and with different tokenizers. We study Allspark at two scales: controlled Qwen experiments across math and reasoning, and larger-scale Inkling experiments on ARC-AGI-2. The Inkling experiments show accuracy gains in within-family and cross-family settings, including transfer to Kimi and Nemotron, with benefits that vary across inference settings. These findings motivate reusing a trained weak teacher across strong students and examining the resulting accuracy--token tradeoff.