mlsanityData to 7 Oct 2026

Mode Connectivity Beyond Classifiers: Evidence from Generative and Contrastive Models

arXiv 2608.30366 ↗First posted 31 Aug 2026cs.LGcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The loss landscape of Deep Neural Networks (DNNs) exhibits highly complex and non-convex properties. Recent studies have revealed the phenomenon of mode connectivity, demonstrating that independently trained network modes can be connected via a continuous low-loss path. However, existing mode connectivity research is predominantly confined to classifier-based models, leaving it an open question whether similar geometric properties exist in modern complex models. In this paper, we extend the boundaries of mode connectivity to generative and contrastive domains (specifically DDPM and NanoCLIP). Addressing the unique architecture of DDPM and CLIP, we propose an architecture-aware connection building algorithm. Extensive empirical results demonstrate for the first time that we successfully discover mode connectivity between independently trained DDPM and NanoCLIP modes. Our work provides a novel perspective for understanding the geometric properties of the loss landscapes in modern generative and contrastive models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 18 of its 29 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

18 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  7. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  8. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  9. 2020Designing Network Design Spacescited by 954
  10. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 585
  11. 2017Deep Layer Aggregationcited by 436
  12. 2018Essentially No Barriers in Neural Network Energy Landscapecited by 328
5 more
  1. 2021The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networkscited by 285
  2. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 261
  3. 2020Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernelcited by 185
  4. 2024Vanishing Variance Problem in Fully Decentralized Neural-Network Systemscited by 3
  5. 2025Connecting Independently Trained Modes via Layer-Wise Connectivitycited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025On Linear Mode Connectivity of Mixture-of-Experts Architectures
  2. 2026Traversing the solution space of neural networks with Hessian Null Space Continuation
  3. 2025The GAN is dead; long live the GAN! A Modern GAN Baseline
  4. 2025A Theoretical Analysis of Discrete Flow Matching Generative Models
  5. 2024What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models
  6. 2026Bridge Graphical Models: Coupling, Projection, and Current-Preserving Dynamics for Generative Modeling
  7. 2023Unsupervised Discovery of Semantic Latent Directions in Diffusion Models
  8. 2023Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models