Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories
Not on the map. Why?
Abstract (arXiv)
Diffusion and flow-based generative models produce strong images, yet their controllability remains largely endpoint-centric: users specify conditions and receive final outputs, while the intermediate generative dynamics remain hidden. Recent methods have begun to exploit generation order and process decomposition to improve sample quality, but still treat intermediate states as internal computation rather than objects for interaction. We propose Trajectory Forcing (TF), a trajectory-centric framework that makes the generation path explicit, semantic, and editable. TF organizes synthesis as a sequence of semantically structured stages, progressing from global layout to object-, part-, and detail-level representations. Each stage produces a decodable latent state that can be inspected, evaluated, and locally edited before the next stage begins. To instantiate this path, we derive coarse-to-fine teacher hierarchies by clustering pretrained visual representations such as DINOv2, and train a hierarchy-conditioned one-step flow-matching model at each level. We further introduce trajectory-aware metrics that measure structural consistency and local controllability beyond endpoint quality metrics such as FID. Experiments show that TF achieves competitive sample quality while exposing coherent intermediate states and supporting localized edits across semantic levels. By shifting the focus from final images to the generative path itself, TF opens a route toward controllable, trajectory-aware image synthesis.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 48 of its 54 references that are in the map. First posted on arXiv 21 Jun 2026.
Builds on
48 of 54 references.
References · 48
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
- 2016Improved Techniques for Training GANscited by 2,970
- 2023Consistency Modelscited by 1,822
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2025SAM 3: Segment Anything with Conceptscited by 906
- 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
- 2022On Distillation of Guided Diffusion Modelscited by 725
36 more
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2024Autoregressive Image Generation without Vector Quantizationcited by 651
- 2025Mean Flows for One-step Generative Modelingcited by 561
- 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
- 2023Improved Techniques for Training Consistency Modelscited by 439
- 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 402
- 2024One Step Diffusion via Shortcut Modelscited by 391
- 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
- 2025Diffusion Transformers with Representation Autoencoderscited by 295
- 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 283
- 2022The Role of ImageNet Classes in Fr'echet Inception Distancecited by 213
- 2019Effectively Unbiased FID and Inception Score and where to find themcited by 165
- 2026Generative Modeling via Driftingcited by 116
- 2025Inductive Moment Matchingcited by 113
- 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 110
- 2023GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphscited by 97
- 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 97
- 2025PixNerd: Pixel Neural Field Diffusioncited by 82
- 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 80
- 2023Diffusion Models as Masked Autoencoderscited by 71
- 2025DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generationcited by 60
- 2026One-step Latent-free Image Generation with Pixel Mean Flowscited by 53
- 2025AlphaFlow: Understanding and Improving MeanFlow Modelscited by 41
- 2026Representation Fr'echet Loss for Visual Generationcited by 37
- 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 36
- 2025Transition Models: Rethinking the Generative Learning Objectivecited by 34
- 2026Self-Supervised Flow Matching for Scalable Multi-Modal Synthesiscited by 31
- 2025Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learningcited by 21
- 2024Diffusion Posterior Sampling is Computationally Intractablecited by 20
- 2024Denoising Autoregressive Representation Learningcited by 9
- 2025MeanFlow Transformers with Representation Autoencoderscited by 6
- 2026VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenizationcited by 6
- 2025Next Visual Granularity Generationcited by 5
- 2021Causal Representation Learning for Context-Aware Face Transfercited by 2
- 2026Slots, Transitions, Loops: Learning Composable World Models for ARCcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025EnfoPath: Energy-Informed Analysis of Generative Trajectories in Flow Matching
- 2026Diffusion Trajectory Modeling for Semantic Correspondence
- 2026Glance Forcing: Making Bidirectional Diffusion Autoregressive at a Glance
- 2026Reward-Aware Trajectory Shaping for Few-step Visual Generation
- 2026Diffusion Path Alignment for Long-Range Motion Generation and Domain Transitions
- 2026A Kinetic Energy Perspective of Flow Matching
- 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild
- 2026Contextual Flow Matching: Adaptive Step Selection in Flow Models for Efficient Visual Generation