# Representation Fr'echet Loss for Visual Generation

arXiv: https://arxiv.org/abs/2604.28190 · HTML: https://mlsanity.com/paper/2604.28190/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 74 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Builds on (most cited first, 30 of 51)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models](https://mlsanity.com/paper/2501.01423/) (2025)
- [BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset](https://mlsanity.com/paper/2505.09568/) (2025)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models](https://mlsanity.com/paper/2410.11081/) (2024)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)

## Built on by (most cited first, 30 of 37)

- [Improved Baselines with Representation Autoencoders](https://mlsanity.com/paper/2605.18324/) (2026)
- [MIND: Monge Inception Distance for Generative Models Evaluation](https://mlsanity.com/paper/2605.06797/) (2026)
- [Diffusing in the Right Space: A Systematic Study of Latent Diffusability](https://mlsanity.com/paper/2606.03578/) (2026)
- [DiffusionBench: On Holistic Evaluation of Diffusion Transformers](https://mlsanity.com/paper/2606.24888/) (2026)
- [Three-Body Scattering for Generative Modeling](https://mlsanity.com/paper/2607.18198/) (2026)
- [Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation](https://mlsanity.com/paper/2607.27372/) (2026)
- [AdvFD: Boosting Visual Generation via Adversarial Fr'echet Distance Loss](https://mlsanity.com/paper/2608.11205/) (2026)
- [Normalizing Trajectory Models](https://mlsanity.com/paper/2605.08078/) (2026)
- [Trajectory Forcing: Structure-First Generation with Controllable Semantic Trajectories](https://mlsanity.com/paper/2606.22527/) (2026)
- [Amortized Moment Matching for Visual Generation](https://mlsanity.com/paper/2607.26860/) (2026)
- [GenFirst: Generation Before Reconstruction for Stable End-to-End Latent Generative Modeling](https://mlsanity.com/paper/2608.29335/) (2026)
- [MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation](https://mlsanity.com/paper/2606.26016/) (2026)
- [Improving Medical Image Generative Models with Fr'echet Distance Loss](https://mlsanity.com/paper/2607.13300/) (2026)
- [Beyond Token-Level Cross-Entropy: Fr'echet Distributional Post-Training for Autoregressive Image Generation](https://mlsanity.com/paper/2608.00562/) (2026)
- [Unifying Distributional Training for One-Step Visual Generation](https://mlsanity.com/paper/2609.35763/) (2026)
- [Improved Distributional Diffusion Models](https://mlsanity.com/paper/2609.37147/) (2026)
- [Diffusion Image Generation with Explicit Modeling of Data Manifold Geometry](https://mlsanity.com/paper/2606.00094/) (2026)
- [Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation](https://mlsanity.com/paper/2608.14172/) (2026)
- [A Latent Distribution Perspective on Evaluating and Improving Latent Generative Models](https://mlsanity.com/paper/2609.24088/) (2026)
- [DreamStream: Towards Policy-Oriented Generative Simulation for End-to-End Driving](https://mlsanity.com/paper/2609.26792/) (2026)
- [ViRDM: Taming Representation Distribution Matching for Few-Step Causal Video Generation](https://mlsanity.com/paper/2609.28923/) (2026)
- [An End-to-End Latent-Rollout Approach for Pushing Few-Step ImageNet-256 Generation to FID 1.11 without Fr'echet Losses](https://mlsanity.com/paper/2609.32376/) (2026)
- [RepFlow: Reciprocal Supervision Improves Generation and Representation in Flow Models](https://mlsanity.com/paper/2609.33217/) (2026)
- [Coherence-Aware Distributional Evaluation of Open-Ended Text Generation](https://mlsanity.com/paper/2609.34240/) (2026)
- [What Visual Generators Need from Teachers: Rethinking Representation Alignment](https://mlsanity.com/paper/2609.34732/) (2026)
- [From Scores to Samples: Elastic Forcing for Autoregressive Video Generation](https://mlsanity.com/paper/2609.35491/) (2026)
- [Improving Generative Model Self-Training with Geometrically Modified Outputs](https://mlsanity.com/paper/2609.35512/) (2026)
- [HiRAE: Hierarchical Representation Autoencoding with Residual Budgets](https://mlsanity.com/paper/2609.37775/) (2026)
- [DMA2: Pixel-space Distribution Matching with Adversarial and Anchor Losses](https://mlsanity.com/paper/2609.38156/) (2026)
- [Visualizing Distribution Coverage in Generative Diffusion Models](https://mlsanity.com/paper/2609.38853/) (2026)

## Abstract (arXiv)

We show that Fr'echet Distance (FD), long considered impractical as a training objective, can in fact be effectively optimized in the representation space. Our idea is simple: decouple the population size for FD estimation (e.g., 50k) from the batch size for gradient computation (e.g., 1024). We term this approach FD-loss. Optimizing FD-loss reveals several surprising findings. First, post-training a base generator with FD-loss in different representation spaces consistently improves visual quality. Under the Inception feature space, a one-step generator achieves0.72 FID on ImageNet 256x256. Second, the same FD-loss repurposes multi-step generators into strong one-step generators without teacher distillation, adversarial training or per-sample targets. Third, FID can misrank visual quality: modern representations can yield better samples despite worse Inception FID. This motivates FDrk, a multi-representation metric. We hope this work will encourage further exploration of distributional distances in diverse representation spaces as both training objectives and evaluation metrics for generative models.