Meltdown: Circuits and Bifurcations in Point-Cloud-Conditioned 3D Diffusion Transformers
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
Sparse point clouds are a common input modality for 3D surface reconstruction, including in safety-critical settings such as surgical navigation and autonomous perception. Recent point-cloud-conditioned 3D diffusion transformers achieve state-of-the-art results in this regime by leveraging learned priors. We show that these models can fail catastrophically under realistic input variation, and present a mechanistic case study of why. We identify a failure mode we call Meltdown: tiny on-surface perturbations to a sparse input point cloud can fracture the reconstructed output into hundreds of disconnected pieces. Adversarial search recovers Meltdown in 89.9-100% of shapes across the two open-weight state-of-the-art architectures we study (WaLa, Make-a-Shape) on real-world datasets (GSO, SimJEB) and under both DDPM and DDIM sampling. We trace Meltdown along the forward pass: it is governed by how uniformly the points are distributed on the surface, faithfully transduced through the point-cloud encoder, and committed by a single early-denoising cross-attention write in the diffusion backbone. Diffusion-trajectory ensembles exhibit symmetry-breaking near this commit step, consistent with a bifurcation of the reverse process. Through a suite of matched-magnitude controls, we show that the variable on which the model commits is directional, concentrated in a low-rank subspace of the write's perturbation drift. Motivated by this finding, we introduce PowerRemap, a test-time control that reshapes the singular spectrum of the localized write to suppress this drift, with rescue rates of 98.3% on WaLa and 84.6% on Make-a-Shape. Together, these results link a circuit-level cross-attention mechanism to a trajectory-level account of the failure, demonstrating how mechanistic analysis can explain and guide behavior in conditional diffusion transformers.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 28 of its 59 references that are in the map. First posted on arXiv 11 Feb 2026.
Builds on
28 of 59 references.
Closest ancestors · 1
Same direction, newest first
Other references · 27
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
- 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
- 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 422
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
- 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 236
15 more
- 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
- 2024How to use and interpret activation patchingcited by 214
- 2025Open Problems in Mechanistic Interpretabilitycited by 179
- 2019PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Cloudscited by 99
- 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generationcited by 57
- 2022Surface Reconstruction from Point Clouds: A Survey and a Benchmarkcited by 41
- 2024One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Modelscited by 33
- 2024Make-A-Shape: a Ten-Million-scale 3D Shape Modelcited by 25
- 2025Emergence and Evolution of Interpretable Concepts in Diffusion Modelscited by 22
- 2025Hunyuan3D-Omni: A Unified Framework for Controllable Generation of 3D Assetscited by 22
- 2023A Geometric Perspective on Diffusion Modelscited by 18
- 2025The Information Dynamics of Generative Diffusioncited by 13
- 2023A Survey and Benchmark of Automatic Surface Reconstruction from Point Cloudscited by 11
- 2025Interpreting Large Text-to-Image Diffusion Models with Dictionary Learningcited by 6
- 2024Deep Learning for 3D Point Cloud Enhancement: A Surveycited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2023Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generation
- 2025Consistency Diffusion Models for Single-Image 3D Reconstruction with Priors
- 2024PointInfinity: Resolution-Invariant Point Diffusion Models
- 2024Point Cloud Resampling with Learnable Heat Diffusion
- 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation
- 2024Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features
- 2025SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images
- 2023CCD-3DR: Consistent Conditioning in Diffusion for Single-Image 3D Reconstruction