# Uncovering Hidden Subspaces in Video Diffusion Models Using Re-Identification

arXiv: https://arxiv.org/abs/2411.04956 · HTML: https://mlsanity.com/paper/2411.04956/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 33 of its 43 references that are in the map.
- First posted on arXiv 7 Nov 2024.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models](https://mlsanity.com/paper/2306.04675/) (2023)
- [Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation](https://mlsanity.com/paper/2304.08477/) (2023)
- [Differentially Private Diffusion Models](https://mlsanity.com/paper/2210.09929/) (2022)
- [Generative models improve fairness of medical classifiers under distribution shifts](https://mlsanity.com/paper/2304.09218/) (2023)
- [Diffusion Probabilistic Models beat GANs on Medical Images](https://mlsanity.com/paper/2212.07501/) (2022)
- [On the Generalization Properties of Diffusion Models](https://mlsanity.com/paper/2311.01797/) (2023)
- [Deep Learning-based Patient Re-identification Is able to Exploit the Biometric Nature of Medical Chest X-ray Data](https://mlsanity.com/paper/2103.08562/) (2021)
- [Can segmentation models be trained with fully synthetically generated data?](https://mlsanity.com/paper/2209.08256/) (2022)
- [Ultrasound Video Transformers for Cardiac Ejection Fraction Estimation](https://mlsanity.com/paper/2107.00977/) (2021)
- [Feature-Conditioned Cascaded Video Diffusion Models for Precise Echocardiogram Synthesis](https://mlsanity.com/paper/2303.12644/) (2023)
- [D'ARTAGNAN: Counterfactual Video Generation](https://mlsanity.com/paper/2206.01651/) (2022)
- [MGGAN: Solving Mode Collapse using Manifold Guided Training](https://mlsanity.com/paper/1804.04391/) (2018)
- [EchoNet-Synthetic: Privacy-preserving Video Generation for Safe Medical Data Sharing](https://mlsanity.com/paper/2406.00808/) (2024)

## Built on by (most cited first, 2 of 2)

- [Image Generation Diversity Issues and How to Tame Them](https://mlsanity.com/paper/2411.16171/) (2024)
- [Enabling PSO-Secure Synthetic Data Sharing Using Diversity-Aware Diffusion Models](https://mlsanity.com/paper/2506.17975/) (2025)

## Abstract (arXiv)

Latent Video Diffusion Models can easily deceive casual observers and domain experts alike thanks to the produced image quality and temporal consistency. Beyond entertainment, this creates opportunities around safe data sharing of fully synthetic datasets, which are crucial in healthcare, as well as other domains relying on sensitive personal information. However, privacy concerns with this approach have not fully been addressed yet, and models trained on synthetic data for specific downstream tasks still perform worse than those trained on real data. This discrepancy may be partly due to the sampling space being a subspace of the training videos, effectively reducing the training data size for downstream models. Additionally, the reduced temporal consistency when generating long videos could be a contributing factor. In this paper, we first show that training privacy-preserving models in latent space is computationally more efficient and generalize better. Furthermore, to investigate downstream degradation factors, we propose to use a re-identification model, previously employed as a privacy preservation filter. We demonstrate that it is sufficient to train this model on the latent space of the video generator. Subsequently, we use these models to evaluate the subspace covered by synthetic video datasets and thus introduce a new way to measure the faithfulness of generative machine learning models. We focus on a specific application in healthcare echocardiography to illustrate the effectiveness of our novel methods. Our findings indicate that only up to 30.8% of the training videos are learned in latent video diffusion models, which could explain the lack of performance when training downstream tasks on synthetic data.