# Restoring Initial Noise Sensitivity in Text-to-Image Distillation via Geometric Alignment

arXiv: https://arxiv.org/abs/2606.01651 · HTML: https://mlsanity.com/paper/2606.01651/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 65 references that are in the map.
- First posted on arXiv 1 Jun 2026.

## Builds on (most cited first, 8 of 8)

- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [SDXL-Lightning: Progressive Adversarial Diffusion Distillation](https://mlsanity.com/paper/2402.13929/) (2024)
- [Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping](https://mlsanity.com/paper/2402.19159/) (2024)
- [Distilling Diversity and Control in Diffusion Models](https://mlsanity.com/paper/2503.10637/) (2025)
- [Few-Step Diffusion via Score identity Distillation](https://mlsanity.com/paper/2505.12674/) (2025)
- [Score Distillation of Flow Matching Models](https://mlsanity.com/paper/2509.25127/) (2025)

## Abstract (arXiv)

Generative distillation significantly accelerates text-to-image (T2I) generation by compressing multi-step trajectories into few-step student models while preserving perceptual quality. However, existing methods primarily optimize efficiency and output fidelity, often neglecting critical properties of the original trajectory. In this work, we identify a key missing property: sensitivity to initial noise, whose degradation impairs downstream control methods relying on noise-based optimization and manipulation. We trace this issue to standard distillation objectives that enforce pointwise output alignment, inadvertently flattening the input-output landscape and suppressing the teacher's local geometric structure. To address this, we propose Geometry-Aware Distillation (GAD), a sensitivity-preserving framework that aligns the local functional behavior of teacher and student models. Specifically, GAD matches Jacobian-vector products with respect to input noise, enabling the student to reproduce the teacher's differential response to perturbations. Extensive experiments across multiple T2I paradigms and noise-driven control tasks demonstrate that GAD significantly restores sensitivity and improves diversity while maintaining high visual fidelity. Code is available at https://github.com/Hannah1102/GAD.