mlsanityData to 14 Oct 2026

PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Prediction

arXiv 2503.08594 ↗First posted 11 Mar 2025cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Autoregressive point cloud generation has long lagged behind diffusion-based approaches in quality. The performance gap stems from the fact that autoregressive models impose an artificial ordering on inherently unordered point sets, forcing shape generation to proceed as a sequence of local predictions. This sequential bias emphasizes short-range continuity but undermines the model's capacity to capture long-range dependencies, hindering its ability to enforce global structural properties such as symmetry, consistent topology, and large-scale geometric regularities. Inspired by the level-of-detail (LOD) principle in shape modeling, we propose PointNSP, a coarse-to-fine generative framework that preserves global shape structure at low resolutions and progressively refines fine-grained geometry at higher scales through a next-scale prediction paradigm. This multi-scale factorization aligns the autoregressive objective with the permutation-invariant nature of point sets, enabling rich intra-scale interactions while avoiding brittle fixed orderings. Experiments on ShapeNet show that PointNSP establishes state-of-the-art (SOTA) generation quality for the first time within the autoregressive paradigm. In addition, it surpasses strong diffusion-based baselines in parameter, training, and inference efficiency. Finally, in dense generation with 8,192 points, PointNSP's advantages become even more pronounced, underscoring its scalability potential.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 68 of its 79 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

68 of 79 references.

References · 68

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  9. 2017Neural Discrete Representation Learningcited by 4,712
  10. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  11. 2018Neural Ordinary Differential Equationscited by 3,779
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
38 more
  1. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  3. 2015Variational Inference with Normalizing Flowscited by 2,170
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  5. 2020Point Transformercited by 1,470
  6. 2017Deep Setscited by 1,421
  7. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  8. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  9. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  10. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  11. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  12. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  13. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  14. 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
  15. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  16. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  17. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  18. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  19. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  20. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
  21. 2018GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Modelscited by 413
  22. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
  23. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  24. 2020GraphAF: a Flow-based Autoregressive Model for Molecular Graph Generationcited by 305
  25. 2016Revisiting Classifier Two-Sample Testscited by 268
  26. 2020Learning Gradient Fields for Shape Generationcited by 245
  27. 2018Molecular Transformer - A Model for Uncertainty-Calibrated Chemical Reaction Predictioncited by 229
  28. 2018PU-Net: Point Cloud Upsampling Networkcited by 209
  29. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  30. 2019Scaling Autoregressive Video Modelscited by 190
  31. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  32. 2019PU-GAN: a Point Cloud Upsampling Adversarial Networkcited by 164
  33. 2023PointGPT: Auto-regressively Generative Pre-training from Point Cloudscited by 115
  34. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generationcited by 110
  35. 2022ShapeFormer: Transformer-based Shape Completion via Sparse Representationcited by 106
  36. 2018PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attentioncited by 105
  37. 2019PU-GCN: Point Cloud Upsampling using Graph Convolutional Networkscited by 101
  38. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 98

50 most cited of 68 All 68 (JSON)

Built on by

Not cited yet.