mlsanityData to 14 Oct 2026

CapField-OPD: Learning Continuous Capability Fields via Joint-Anchored Multi-Teacher On-Policy Distillation for Flow Models

arXiv 2609.34658 ↗First posted 28 Sep 2026cs.CVcited by 0

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

Reward-specialized post-training produces strong experts for flow-based generative models, while multi-teacher on-policy distillation (OPD) consolidates their capabilities into a single student. Existing methods, however, route each prompt to a single teacher according to its semantic category, implicitly binding the desired capability to prompt content. This coupling makes capability invocation vulnerable to prompt perturbations and prevents users from explicitly adjusting the strength of the desired capability at inference time. In this work, we introduce CapField-OPD, an OPD framework that integrates multiple teachers into a continuous capability field through explicit capability coordinates. We use teacher models as anchors to construct this field, with the coordinates determining how their outputs are combined. Each capability configuration thus receives a unique supervision target, and capability control no longer depends on prompt semantics. Since the training anchors may not be optimal at inference time, we further profile the learned field on a small calibration set. The coordinate with the highest mean reward serves as the recommended default, while coordinates that are frequently optimal offer a promising candidate set for test-time scaling. Extensive experiments on compositional generation, text rendering, and visual aesthetics demonstrate that CapField-OPD consolidates multiple specialized teachers into a single student while preserving or surpassing their performance, reliably invokes the desired capabilities under semantics-preserving prompt variations, and supports continuous capability control and coordinate-based test-time scaling.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 36 of its 38 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

36 of 38 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026DreOPD: Degraded-Reference Extrapolative On-Policy Distillation for Flow-matching Models10 Aug
  2. 2026Any-OPD: Heterogeneous On-Policy Distillation for Flow-Matching Models via Representation-Space Bridging4 Aug
  3. 2026Flow-OPD: On-Policy Distillation for Flow Matching Models8 May

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2025Qwen-Image Technical Reportcited by 1,044
  6. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  7. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  8. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  9. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  10. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  11. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  12. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 684
21 more
  1. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  2. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  3. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 413
  4. 2025Improving Video Generation with Human Feedbackcited by 284
  5. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
  6. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
  7. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
  8. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 211
  9. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 178
  10. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 87
  11. 2025Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Modelscited by 67
  12. 2025Coefficients-Preserving Sampling for Reinforcement Learning with Flow Matchingcited by 56
  13. 2026DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Modelscited by 40
  14. 2025Fine-Grained GRPO for Precise Preference Alignment in Flow Modelscited by 21
  15. 2025TwinFlow: Realizing One-step Generation on Large Models with Self-adversarial Flowscited by 17
  16. 2026DanceOPD: On-Policy Generative Field Distillationcited by 14
  17. 2026PromptRL: Prompt Matters in RL for Flow-Based Image Generationcited by 12
  18. 2026TMPO: Trajectory Matching Policy Optimization for Diverse and Efficient Diffusion Alignmentcited by 4
  19. 2026Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillationcited by 4
  20. 2026From Sparse to Dense: Multi-View GRPO for Flow Models via Augmented Condition Spacecited by 3
  21. 2026Pave-GRPO: Beyond Instantaneous Guidance through Principled Average Velocity Decompositioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation
  2. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
  3. 2026Recovering General Capabilities via Uncertainty-Calibrated Multi-Teacher On-Policy Distillation
  4. 2026Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation
  5. 2026Self-OPD: On-Policy Distillation for Flow Matching Models without Teacher
  6. 2026Open-MOPD: Diagnosing and Fixing Capability Imbalance in Multi-Teacher On-Policy Distillation
  7. 2026Slow-Fast Multi-Teacher On-Policy Distillation for Capability Preservation
  8. 2026Rethinking Self-Distillation for Multi-Teacher Capability Merging