# Accelerating Visual On-Policy Distillation with Batched Speculative Jacobi Rollouts

arXiv: https://arxiv.org/abs/2608.18183 · HTML: https://mlsanity.com/paper/2608.18183/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 24 of its 25 references that are in the map.
- First posted on arXiv 18 Aug 2026.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 24 of 24)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [Accelerating Auto-regressive Text-to-Image Generation with Training-free Speculative Jacobi Decoding](https://mlsanity.com/paper/2410.01699/) (2024)
- [LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding](https://mlsanity.com/paper/2410.03355/) (2024)
- [ZipAR: Parallel Auto-regressive Image Generation through Spatial Locality](https://mlsanity.com/paper/2412.04062/) (2024)
- [Continuous Speculative Decoding for Autoregressive Image Generation](https://mlsanity.com/paper/2411.11925/) (2024)
- [Grouped Speculative Decoding for Autoregressive Image Generation](https://mlsanity.com/paper/2508.07747/) (2025)
- [LANTERN++: Enhancing Relaxed Speculative Decoding with Static Tree Drafting for Visual Auto-regressive Models](https://mlsanity.com/paper/2502.06352/) (2025)
- [Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding](https://mlsanity.com/paper/2604.26779/) (2026)
- [Speculative Jacobi-Denoising Decoding for Accelerating Autoregressive Text-to-image Generation](https://mlsanity.com/paper/2510.08994/) (2025)
- [Speculative Coupled Decoding for Training-Free Lossless Acceleration of Autoregressive Visual Generation](https://mlsanity.com/paper/2510.24211/) (2025)
- [Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem](https://mlsanity.com/paper/2510.22876/) (2025)
- [SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image Generation](https://mlsanity.com/paper/2512.07503/) (2025)
- [Parallel Jacobi Decoding for Fast Autoregressive Image Generation](https://mlsanity.com/paper/2606.05703/) (2026)
- [Knowledge Distillation for Visual Autoregressive Models](https://mlsanity.com/paper/2606.06078/) (2026)
- [SJD-PV: Speculative Jacobi Decoding with Phrase Verification for Autoregressive Image Generation](https://mlsanity.com/paper/2603.06666/) (2026)
- [SJD-PAC: Accelerating Speculative Jacobi Decoding via Proactive Drafting and Adaptive Continuation](https://mlsanity.com/paper/2603.18599/) (2026)
- [SJD-VP: Speculative Jacobi Decoding with Verification Prediction for Autoregressive Image Generation](https://mlsanity.com/paper/2603.27115/) (2026)
- [PathRelax: Parallel-Path Relaxed Speculative Jacobi Decoding for Accelerating Auto-Regressive Text-to-Image Generation](https://mlsanity.com/paper/2606.10492/) (2026)

## Abstract (arXiv)

Visual on-policy distillation (OPD) improves the training of compact visual autoregressive models by learning from trajectories generated by the current student. However, these online rollouts are still produced token by token with autoregressive decoding, which adds substantial cost to every on-policy training step. Speculative Jacobi Decoding (SJD) provides an alternative because it can process multiple tokens in parallel without an auxiliary draft model, but the original method is designed for single-sequence inference. We introduce HB-SJD, a batched SJD rollout backend for visual OPD. HB-SJD allows each image to advance independently according to its own decoding progress, while images at different sequence positions are still verified in batched model forwards. As images finish, HB-SJD switches between Full and Compact execution to reduce the cost of later rollout rounds. HB-SJD only replaces the student rollout backend and leaves the teacher, distillation objective, and optimization procedure unchanged. Experiments with LlamaGen show that HB-SJD substantially reduces rollout and end-to-end training time while preserving the generation quality of the distilled student.