mlsanityData to 7 Oct 2026

The Hessian Penalty: A Weak Prior for Unsupervised Disentanglement

arXiv 2008.10599 ↗First posted 24 Aug 2020cs.CV cs.GR cs.LGcited by 90

Not on the map. Why?

Abstract (arXiv)

Existing disentanglement methods for deep generative models rely on hand-picked priors and complex encoder-based architectures. In this paper, we propose the Hessian Penalty, a simple regularization term that encourages the Hessian of a generative model with respect to its input to be diagonal. We introduce a model-agnostic, unbiased stochastic approximation of this term based on Hutchinson's estimator to compute it efficiently during training. Our method can be applied to a wide range of deep generators with just a few lines of code. We show that training with the Hessian Penalty often causes axis-aligned disentanglement to emerge in latent space when applied to ProGAN on several datasets. Additionally, we use our regularization term to identify interpretable directions in BigGAN's latent space in an unsupervised fashion. Finally, we provide empirical evidence that the Hessian Penalty encourages substantial shrinkage when applied to over-parameterized latent spaces.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 21; 2023: 19; 2024: 9; 2025: 7; 2026: 3.

90 papers.

Sort:

By area:

  1. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  2. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
  3. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  4. 2021Counterfactual Generative Networkscited by 111
  5. 20213D-aware Image Synthesis via Learning Structural and Textural Representationscited by 109
  6. 2021Independent mechanism analysis, a new concept?cited by 96
  7. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  8. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  9. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  10. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  11. 2021Discover the Unknown Biased Attribute of an Image Classifiercited by 46
  12. 2021Learning Disentangled Representations in the Imaging Domaincited by 46
38 more
  1. 2021Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGANcited by 45
  2. 2021Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Modelscited by 44
  3. 2021Discovering and Explaining the Representation Bottleneck of DNNscited by 42
  4. 2021ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generatorscited by 40
  5. 2023Disentanglement via Latent Quantizationcited by 40
  6. 2022Spatially-Adaptive Multilayer Selection for GAN Inversion and Editingcited by 39
  7. 2021Efficient Iterative Amortized Inference for Learning Symmetric and Disentangled Multi-Object Representationscited by 36
  8. 2021Orthogonal Jacobian Regularization for Unsupervised Disentanglement in Image Generationcited by 35
  9. 2022BlobGAN: Spatially Disentangled Scene Representationscited by 35
  10. 2021Video Autoencoder: self-supervised disentanglement of static 3D structure and motioncited by 34
  11. 2023Measuring axiomatic soundness of counterfactual image modelscited by 34
  12. 2022StyleT2I: Toward Compositional and High-Fidelity Text-to-Image Synthesiscited by 33
  13. 2024Disentangled 3D Scene Generation with Layout Learningcited by 33
  14. 2021Multi-Facet Clustering Variational Autoencoderscited by 28
  15. 2021Where and What? Examining Interpretable Disentangled Representationscited by 27
  16. 2021Commutative Lie Group VAE for Disentanglement Learningcited by 27
  17. 2021UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Modelscited by 23
  18. 2025Personalized Image Generation with Deep Generative Models: A Decade Surveycited by 20
  19. 2023Latent Traversals in Generative Models as Potential Flowscited by 17
  20. 2022Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Biascited by 16
  21. 2021Toward a Visual Concept Vocabulary for GAN Latent Spacecited by 15
  22. 2022Attribute Group Editing for Reliable Few-shot Image Generationcited by 15
  23. 2023LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesiscited by 15
  24. 2021Hallucinating Pose-Compatible Scenescited by 14
  25. 2021Disentangled Representations from Non-Disentangled Modelscited by 13
  26. 2023Domain Expansion of Image Generatorscited by 13
  27. 2024Tripod: Three Complementary Inductive Biases for Disentangled Representation Learningcited by 13
  28. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  29. 2024Compositional Image Decomposition with Diffusion Modelscited by 12
  30. 2021Latent Transformations via NeuralODEs for GAN-based Image Editingcited by 11
  31. 2023Flow Factorized Representation Learningcited by 11
  32. 2021DiagViB-6: A Diagnostic Benchmark Suite for Vision Models in the Presence of Shortcut and Generalization Opportunitiescited by 10
  33. 2022Mind the Gap in Distilling StyleGANscited by 10
  34. 2021Model-centric Data Manifold: the Data Through the Eyes of the Modelcited by 9
  35. 2022Review of Disentanglement Approaches for Medical Applications -- Towards Solving the Gordian Knot of Generative Models in Healthcarecited by 9
  36. 2022Training and Tuning Generative Neural Radiance Fields for Attribute-Conditional 3D-Aware Face Generationcited by 8
  37. 2023That's What I Said: Fully-Controllable Talking Face Generationcited by 8
  38. 2023NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigationcited by 8

50 most cited of 90 All 90 (JSON)