# WiT: Waypoint Diffusion Transformers for Alleviating Trajectory Conflict in Pixel-Space Image Generation

arXiv: https://arxiv.org/abs/2603.15132 · HTML: https://mlsanity.com/paper/2603.15132/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 35 of its 61 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 35)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022)
- [Finite Scalar Quantization: VQ-VAE Made Simple](https://mlsanity.com/paper/2309.15505/) (2023)
- [Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models](https://mlsanity.com/paper/2404.07724/) (2024)
- [REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers](https://mlsanity.com/paper/2504.10483/) (2025)
- [Scalable Adaptive Computation for Iterative Generation](https://mlsanity.com/paper/2212.11972/) (2022)
- [Return of Unconditional Generation: A Self-supervised Representation Generation Method](https://mlsanity.com/paper/2312.03701/) (2023)
- [Latent Diffusion Model without Variational Autoencoder](https://mlsanity.com/paper/2510.15301/) (2025)
- [What matters for Representation Alignment: Global Information or Spatial Structure?](https://mlsanity.com/paper/2512.10794/) (2025)
- [PixelDiT: Pixel Diffusion Transformers for Image Generation](https://mlsanity.com/paper/2511.20645/) (2025)
- [Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion](https://mlsanity.com/paper/2410.19324/) (2024)
- [DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation](https://mlsanity.com/paper/2511.19365/) (2025)

## Built on by (most cited first, 1 of 1)

- [Representative Attention For Vision Transformers](https://mlsanity.com/paper/2605.14913/) (2026)

## Abstract (arXiv)

While recent Flow Matching models avoid the reconstruction bottlenecks of latent autoencoders by operating directly in pixel space, the raw pixel manifold provides little explicit semantic organization, making target-specific transport directions difficult for a shared finite-capacity vector field to distinguish. When different transport requirements are locally entangled in this way, their learning signals can interfere and hinder optimization, a phenomenon we refer to as trajectory conflict. To alleviate trajectory conflict while preserving direct generation in pixel space, we propose Waypoint Diffusion Transformers (WiT), which introduces explicit semantic routing into pixel-space generation. WiT structures pixel-space prediction through compact semantic waypoints projected from pre-trained vision representations, providing a semantically organized intermediate representation that complements direct pixel prediction. During ODE integration, a lightweight Waypoints Predictor dynamically infers the semantic waypoint from the current noisy state, and the predicted waypoint continuously conditions the primary Pixel Space Generator through our Just-Pixel AdaLN mechanism. This spatially varying semantic modulation reorganizes the finite-capacity learning problem and helps the model preserve target-specific transport directions while retaining generation entirely in pixel space. Experiments on ImageNet 2012 demonstrate consistent improvements over strong pixel-space baselines across model scales, with WiT-H/16 achieving an FID of 1.79. Code will be released at https://hainuo-wang.github.io/WiT.