# Exploring Diffusion Time-steps for Unsupervised Representation Learning

arXiv: https://arxiv.org/abs/2401.11430 · HTML: https://mlsanity.com/paper/2401.11430/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 21 Jan 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Towards a Definition of Disentangled Representations](https://mlsanity.com/paper/1812.02230/) (2018)
- [Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis](https://mlsanity.com/paper/1911.09267/) (2019)
- [Counterfactual Zero-Shot and Open-Set Visual Recognition](https://mlsanity.com/paper/2103.00887/) (2021)
- [Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models](https://mlsanity.com/paper/2212.12990/) (2022)
- [Counterfactuals uncover the modular structure of deep generative models](https://mlsanity.com/paper/1812.03253/) (2018)
- [Self-Supervised Learning Disentangled Group Representation as Feature](https://mlsanity.com/paper/2110.15255/) (2021)
- [Effective Sparsification of Neural Networks with Global Sparsity Constraint](https://mlsanity.com/paper/2105.01571/) (2021)

## Built on by (most cited first, 30 of 36)

- [Diffusion Models and Representation Learning: A Survey](https://mlsanity.com/paper/2407.00783/) (2024)
- [Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoning](https://mlsanity.com/paper/2505.07538/) (2025)
- [Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards](https://mlsanity.com/paper/2503.11240/) (2025)
- [MaskGWM: A Generalizable Driving World Model with Video Mask Reconstruction](https://mlsanity.com/paper/2502.11663/) (2025)
- [Denoising Task Difficulty-based Curriculum for Training Diffusion Models](https://mlsanity.com/paper/2403.10348/) (2024)
- [Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling](https://mlsanity.com/paper/2502.05743/) (2025)
- [On Denoising Walking Videos for Gait Recognition](https://mlsanity.com/paper/2505.18582/) (2025)
- [Eta Inversion: Designing an Optimal Eta Function for Diffusion-based Real Image Editing](https://mlsanity.com/paper/2403.09468/) (2024)
- [Temporal Score Analysis for Understanding and Correcting Diffusion Artifacts](https://mlsanity.com/paper/2503.16218/) (2025)
- [Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2510.23974/) (2025)
- [Diffusion Bridge AutoEncoders for Unsupervised Representation Learning](https://mlsanity.com/paper/2405.17111/) (2024)
- [Going beyond Compositions, DDPMs Can Produce Zero-Shot Interpolations](https://mlsanity.com/paper/2405.19201/) (2024)
- [Towards Unsupervised Domain Bridging via Image Degradation in Semantic Segmentation](https://mlsanity.com/paper/2412.10339/) (2024)
- [Stem-OB: Generalizable Visual Imitation Learning with Stem-Like Convergent Observation through Diffusion Inversion](https://mlsanity.com/paper/2411.04919/) (2024)
- [Latent Space Disentanglement in Diffusion Transformers Enables Precise Zero-shot Semantic Editing](https://mlsanity.com/paper/2411.08196/) (2024)
- [Exploring the latent space of diffusion models directly through singular value decomposition](https://mlsanity.com/paper/2502.02225/) (2025)
- [Harnessing Text-to-Image Diffusion Models for Point Cloud Self-Supervised Learning](https://mlsanity.com/paper/2507.09102/) (2025)
- [Redefining <Creative> in Dictionary: Towards an Enhanced Semantic Understanding of Creative Generation](https://mlsanity.com/paper/2410.24160/) (2024)
- [DRESS: Disentangled Representation-based Self-Supervised Meta-Learning for Diverse Tasks](https://mlsanity.com/paper/2503.09679/) (2025)
- [JEDI: Joint Embedding Diffusion World Model for Online Model-Based Reinforcement Learning](https://mlsanity.com/paper/2605.13013/) (2026)
- [Latent Space Disentanglement in Diffusion Transformers Enables Zero-shot Fine-grained Semantic Editing](https://mlsanity.com/paper/2408.13335/) (2024)
- [On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning](https://mlsanity.com/paper/2506.00136/) (2025)
- [CoDi: Subject-Consistent and Pose-Diverse Text-to-Image Generation](https://mlsanity.com/paper/2507.08396/) (2025)
- [Characterizing Motion Encoding in Video Diffusion Timesteps](https://mlsanity.com/paper/2512.22175/) (2025)
- [Diffusion Representations for Fine-Grained Image Classification: A Marine Plankton Case Study](https://mlsanity.com/paper/2601.13416/) (2026)
- [Order Is Not Layout: Order-to-Space Bias in Image Generation](https://mlsanity.com/paper/2603.03714/) (2026)
- [Revisiting Diffusion Autoencoder Training for Image Reconstruction Quality](https://mlsanity.com/paper/2504.21368/) (2025)
- [Performance Asymmetry in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2505.19698/) (2025)
- [Controllable Coupled Image Generation via Diffusion Models](https://mlsanity.com/paper/2506.06826/) (2025)
- [CosmoFlow: Scale-Aware Representation Learning for Cosmology with Flow Matching](https://mlsanity.com/paper/2507.11842/) (2025)

## Abstract (arXiv)

Representation learning is all about discovering the hidden modular attributes that generate the data faithfully. We explore the potential of Denoising Diffusion Probabilistic Model (DM) in unsupervised learning of the modular attributes. We build a theoretical framework that connects the diffusion time-steps and the hidden attributes, which serves as an effective inductive bias for unsupervised learning. Specifically, the forward diffusion process incrementally adds Gaussian noise to samples at each time-step, which essentially collapses different samples into similar ones by losing attributes, e.g., fine-grained attributes such as texture are lost with less noise added (i.e., early time-steps), while coarse-grained ones such as shape are lost by adding more noise (i.e., late time-steps). To disentangle the modular attributes, at each time-step t, we learn a t-specific feature to compensate for the newly lost attribute, and the set of all 1,...,t-specific features, corresponding to the cumulative set of lost attributes, are trained to make up for the reconstruction error of a pre-trained DM at time-step t. On CelebA, FFHQ, and Bedroom datasets, the learned feature significantly improves attribute classification and enables faithful counterfactual generation, e.g., interpolating only one specified attribute between two images, validating the disentanglement quality. Codes are in https://github.com/yue-zhongqi/diti.