mlsanityData to 7 Oct 2026

SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data

arXiv 2103.15619 ↗First posted 29 Mar 2021cs.LG cs.CVcited by 78

Not on the map. Why?

Cited by 23 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generative modeling of set-structured data, such as point clouds, requires reasoning over local and global structures at various scales. However, adopting multi-scale frameworks for ordinary sequential data to a set-structured data is nontrivial as it should be invariant to the permutation of its elements. In this paper, we propose SetVAE, a hierarchical variational autoencoder for sets. Motivated by recent progress in set encoding, we build SetVAE upon attentive modules that first partition the set and project the partition back to the original cardinality. Exploiting this module, our hierarchical VAE learns latent variables at multiple scales, capturing coarse-to-fine dependency of the set elements while achieving permutation invariance. We evaluate our model on point cloud generation task and achieve competitive performance to the prior arts with substantially smaller model capacity. We qualitatively demonstrate that our model generalizes to unseen set sizes and learns interesting subset relations without supervision. Our implementation is available at https://github.com/jw9730/setvae.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

26 of 36 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2017Deep Setscited by 1,394
  8. 2020Object-Centric Learning with Slot Attentioncited by 865
  9. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  10. 2016Pixel Recurrent Neural Networkscited by 814
  11. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  12. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
14 more
  1. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  2. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  3. 2016Ladder Variational Autoencoderscited by 315
  4. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  5. 2016Towards a Neural Statisticiancited by 119
  6. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  7. 2018Inferring Semantic Layout for Hierarchical Text-to-Image Synthesiscited by 101
  8. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 97
  9. 2018Point Cloud GANcited by 79
  10. 2019Deep Set Prediction Networkscited by 71
  11. 2019FSPool: Learning Set Representations with Featurewise Sort Poolingcited by 53
  12. 2020Conditional Set Generation with Transformerscited by 32
  13. 2020Exchangeable Neural ODE for Set Modelingcited by 19
  14. 2020Energy-Based Processes for Exchangeable Datacited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 19; 2024: 21; 2025: 19; 2026: 8.

78 papers.

Sort:

By area:

  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  2. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generationcited by 110
  3. 2024Advances in 3D Generation: A Surveycited by 62
  4. 2022Fast Point Cloud Generation with Straight Flowscited by 56
  5. 2021Learning Disentangled Representations in the Imaging Domaincited by 46
  6. 2024RangeLDM: Fast Realistic LiDAR Point Cloud Generationcited by 45
  7. 2022Autoregressive 3D Shape Generation via Canonical Mappingcited by 37
  8. 2022Few-Shot Diffusion Modelscited by 37
  9. 2021Top-N: Equivariant set and graph generation without exchangeabilitycited by 36
  10. 2023CaloClouds: Fast Geometry-Independent Highly-Granular Calorimeter Simulationcited by 33
  11. 20233DQD: Generalized Deep 3D Shape Prior via Part-Discretized Diffusion Processcited by 25
  12. 2023GECCO: Geometrically-Conditioned Point Diffusion Modelscited by 24
38 more
  1. 2023EPiC-GAN: Equivariant Point Cloud Generation for Particle Jetscited by 23
  2. 2024Imagination Policy: Using Generative Point Cloud Models for Learning Manipulation Policiescited by 18
  3. 2025Not-So-Optimal Transport Flows for 3D Point Cloud Generationcited by 17
  4. 20232D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervisioncited by 16
  5. 2024NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generationcited by 13
  6. 2023Pushing the Limits of 3D Shape Generation at Scalecited by 11
  7. 2023MVDD: Multi-View Depth Diffusion Modelscited by 10
  8. 2024PolyOculus: Simultaneous Multi-view Image-based Novel View Synthesiscited by 10
  9. 2024On permutation-invariant neural networkscited by 10
  10. 2023Set-based Neural Network Encoding Without Weight Tyingcited by 9
  11. 2023FrePolad: Frequency-Rectified Point Latent Diffusion for Point Cloud Generationcited by 9
  12. 2023Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generationcited by 9
  13. 2025PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Predictioncited by 9
  14. 2022Conditional set generation using Seq2seq modelscited by 7
  15. 2023Towards End-to-End Generative Modeling of Long Videos with Memory-Efficient Bidirectional Transformerscited by 7
  16. 2024LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Imagecited by 7
  17. 2024Efficient 3D Shape Generation via Diffusion Mamba with Bidirectional SSMscited by 7
  18. 2024Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Modelscited by 7
  19. 2021SCHA-VAE: Hierarchical Context Aggregation for Few-Shot Generationcited by 6
  20. 2024Multi-scale Latent Point Consistency Models for 3D Shape Generationcited by 6
  21. 2025Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matchingcited by 6
  22. 2024A Unified Framework for Human-Allied Learning of Probabilistic Circuitscited by 5
  23. 2024DAGSM: Disentangled Avatar Generation with GS-enhanced Meshcited by 5
  24. 2025Unified Cross-Scale 3D Generation and Understanding via Autoregressive Modelingcited by 5
  25. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 5
  26. 2024Jigsaw++: Imagining Complete Shape Priors for Object Reassemblycited by 4
  27. 2022FusionVAE: A Deep Hierarchical Variational Autoencoder for RGB Image Fusioncited by 3
  28. 2025GrabS: Generative Embodied Agent for 3D Object Segmentation without Scene Supervisioncited by 3
  29. 2025MFM-point: Multi-scale Flow Matching for Point Cloud Generationcited by 3
  30. 2021SHIFT15M: Fashion-specific dataset for set-to-set matching with several distribution shiftscited by 2
  31. 2022SUPA: A Lightweight Diagnostic Simulator for Machine Learning in Particle Physicscited by 2
  32. 2023DeepEMD: A Transformer-based Fast Estimation of the Earth Mover's Distancecited by 2
  33. 2023Diverse Shape Completion via Style Modulated Generative Adversarial Networkscited by 2
  34. 2024Point Cloud Synthesis Using Inner Product Transformscited by 2
  35. 2024Symbolic Disentangled Representations for Imagescited by 2
  36. 2025A Continuous-Time Consistency Model for 3D Point Cloud Generationcited by 2
  37. 2025Scalable Single-Cell Gene Expression Generation with Latent Diffusion Modelscited by 2
  38. 2025Rethinking Metrics and Diffusion Architecture for 3D Point Cloud Generationcited by 2

50 most cited of 78 All 78 (JSON)