mlsanityData to 7 Oct 2026

GaussianAnything: Interactive Point Cloud Flow Matching For 3D Object Generation

arXiv 2411.08033 ↗First posted 12 Nov 2024cs.CV cs.AI cs.GRcited by 12

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

While 3D content generation has advanced significantly, existing methods still face challenges with input formats, latent space design, and output representations. This paper introduces a novel 3D generation framework that addresses these challenges, offering scalable, high-quality 3D generation with an interactive Point Cloud-structured Latent space. Our framework employs a Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal) renderings as input, using a unique latent space design that preserves 3D shape information, and incorporates a cascaded latent flow-based model for improved shape-texture disentanglement. The proposed method, GaussianAnything, supports multi-modal conditional 3D generation, allowing for point cloud, caption, and single image inputs. Notably, the newly proposed latent space naturally enables geometry-texture disentanglement, thus allowing 3D-aware editing. Experimental results demonstrate the effectiveness of our approach on multiple datasets, outperforming existing native 3D methods in both text- and image-conditioned 3D generation.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 93 of its 109 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

93 of 109 references.

Closest ancestors · 18

Same direction, newest first

  1. 2024Atlas Gaussians Diffusion for 3D Generation23 Aug
  2. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
  3. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer23 May
  4. 2024PointInfinity: Resolution-Invariant Point Diffusion Models4 Apr
  5. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors4 Mar
  6. 2023Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation21 Dec
  7. 2023Mosaic-SDF for 3D Generative Models14 Dec
  8. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  9. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  10. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  11. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  12. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization29 Jun
6 more
  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  2. 2023Controllable Mesh Generation Through Sparse Latent Point Diffusion Models14 Mar
  3. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models26 Jan
  4. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  6. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 75

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2022Flow Matching for Generative Modelingcited by 5,426
  11. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  12. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
38 more
  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  9. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  10. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  11. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  12. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  13. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  14. 2020Point Transformercited by 1,462
  15. 2018Demystifying MMD GANscited by 1,268
  16. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  17. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  18. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  19. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  20. 2020On Layer Normalization in the Transformer Architecturecited by 906
  21. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  22. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  23. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  24. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  25. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  26. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  27. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 707
  28. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  29. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  30. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  31. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  32. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
  33. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  34. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  35. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  36. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  37. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  38. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generatorcited by 10
  2. 2026GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generationcited by 4
  3. 2026Generative 3D Gaussians with Learned Density Controlcited by 4
  4. 2025Wukong's 72 Transformations: High-fidelity Textured 3D Morphing via Flow Modelscited by 3
  5. 2026LiFlow: Flow Matching for 3D LiDAR Scene Completioncited by 2
  6. 2025Unifi3D: A Study on 3D Representations for Generation and Reconstruction in a Common Frameworkcited by 1
  7. 2025Object Reconstruction under Occlusion with Generative Priors and Contact-induced Constraintscited by 1
  8. 2026KitchenTwin: Semantically and Geometrically Grounded 3D Kitchen Digital Twinscited by 1
  9. 2026FLUX3D: High-Fidelity 3D Gaussian Generation with Diffusion-Aligned Sparse Representationcited by 0
  10. 2026Visibility-Guided Structured Measure Flow for Class-Conditioned 3D Gaussian Generationcited by 0
  11. 2026OREO: Fidelity Alignment in 3D Generation via On-the-fly Rendering-Editing Optimizationcited by 0
  12. 2026FILIGREE3D: Scaling Sparse Latent Flow Matching for Ultra-High-Resolution Image-to-3D Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Dragen3D: Multiview Geometry Consistent 3D Gaussian Generation with Drag-Based Control
  2. 2025MFM-point: Multi-scale Flow Matching for Point Cloud Generation
  3. 2024Structured 3D Latents for Scalable and Versatile 3D Generation
  4. 2025UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generation
  5. 2026Fusion-Aware Direct 3D Gaussian Generation with Structured Patch Latent Flows
  6. 2025InStyle: Instant Appearance Stylization of 3D Shapes
  7. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets
  8. 2026Class-Partitioned VQ-VAE and Latent Flow Matching for Point Cloud Scene Generation