mlsanityData to 7 Oct 2026

REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment

arXiv 2405.18525 ↗First posted 28 May 2024cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

Traditional image-to-3D models often struggle with scenes containing multiple objects due to biases and occlusion complexities. To address this challenge, we present REPARO, a novel approach for compositional 3D asset generation from single images. REPARO employs a two-step process: first, it extracts individual objects from the scene and reconstructs their 3D meshes using off-the-shelf image-to-3D models; then, it optimizes the layout of these meshes through differentiable rendering techniques, ensuring coherent scene composition. By integrating optimal transport-based long-range appearance loss term and high-level semantic loss term in the differentiable rendering, REPARO can effectively recover the layout of 3D assets. The proposed method can significantly enhance object independence, detail accuracy, and overall scene coherence. Extensive evaluation of multi-object scenes demonstrates that our REPARO offers a comprehensive approach to address the complexities of multi-object 3D scene generation from single images.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 61 of its 70 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

61 of 70 references.

References · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  12. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
38 more
  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  2. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  3. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  4. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  5. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  6. 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 661
  7. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  8. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  9. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  10. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  11. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  12. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  13. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  14. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  15. 2023LangSplat: 3D Language Gaussian Splattingcited by 410
  16. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  17. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  18. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  19. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  20. 2023Text-to-3D using Gaussian Splattingcited by 299
  21. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  22. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  23. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  24. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  25. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  26. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  27. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  28. 2021OTA: Optimal Transport Assignment for Object Detectioncited by 177
  29. 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
  30. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generationcited by 133
  31. 2023Compositional 3D Scene Generation using Locally Conditioned Diffusioncited by 101
  32. 2023GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphscited by 96
  33. 2023Consistent123: Improve Consistency for One Image to 3D Object Synthesiscited by 84
  34. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priorscited by 64
  35. 2024Advances in 3D Generation: A Surveycited by 62
  36. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  37. 2024ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidancecited by 59
  38. 2023Instant3D: Instant Text-to-3D Generationcited by 45

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 20; 2026: 20.

45 papers.

Sort:

By area:

  1. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  2. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
  3. 2025LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPScited by 40
  4. 2025DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusioncited by 23
  5. 2024Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprintcited by 21
  6. 2025MesaTask: Towards Task-Driven Tabletop Scene Generation via 3D Spatial Reasoningcited by 15
  7. 2025SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Modelcited by 15
  8. 2025PAT3D: Physics-Augmented Text-to-3D Scene Generationcited by 14
  9. 2025DanceTogether! Identity-Preserving Multi-Person Interactive Video Generationcited by 12
  10. 2025LAYOUTDREAMER: Physics-guided Layout for Text-to-3D Compositional Scene Generationcited by 11
  11. 2024CompGS: Unleashing 2D Compositionality for Compositional Text-to-3D via Dynamically Optimizing 3D Gaussianscited by 10
  12. 20253D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Frameworkcited by 10
33 more
  1. 2024Semantic Score Distillation Sampling for Compositional Text-to-3D Generationcited by 9
  2. 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 9
  3. 2024AToM: Aligning Text-to-Motion Model at Event-Level with GPT-4Vision Rewardcited by 7
  4. 2025Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimizationcited by 6
  5. 2025I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learnerscited by 6
  6. 2026SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generationcited by 6
  7. 2025Flash Sculptor: Modular 3D Worlds from Objectscited by 5
  8. 2025MoCA: Mixture-of-Components Attention for Scalable Compositional 3D Generationcited by 5
  9. 2025Separate to Collaborate: Dual-Stream Diffusion Model for Coordinated Piano Hand Motion Synthesiscited by 4
  10. 2025InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animationcited by 4
  11. 2025ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editingcited by 4
  12. 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherencecited by 4
  13. 2026SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Imagecited by 4
  14. 2026SceneLinker: Compositional 3D Scene Generation via Semantic Scene Graph from RGB Sequencescited by 3
  15. 2026GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstructioncited by 3
  16. 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstructioncited by 3
  17. 2025DreamAnywhere: Object-Centric Panoramic 3D Scene Generationcited by 2
  18. 2025TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulationcited by 2
  19. 2026PhyMix: Towards Physically Consistent Single-Image 3D Indoor Scene Generation with Implicit--Explicit Optimizationcited by 2
  20. 2026One Video, One World: Turning Monocular Video into Physical 4D Scenescited by 2
  21. 2026DM-CFO: A Diffusion Model for Compositional 3D Tooth Generation with Collision-Free Optimizationcited by 1
  22. 2026Cog2Gen3D: Sculpturing 3D Semantic-Geometric Cognition for 3D Generationcited by 1
  23. 2026Pano3DComposer: Feed-Forward Compositional 3D Scene Generation from Single Panoramic Imagecited by 1
  24. 2026MessyKitchens: Contact-rich object-level 3D scene reconstructioncited by 1
  25. 2026Pose-Aware Diffusion for 3D Generationcited by 1
  26. 2026GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learningcited by 1
  27. 2025CC-FMO: Camera-Conditioned Zero-Shot Single Image to 3D Scene Generation with Foundation Model Orchestrationcited by 0
  28. 2026TIMI: Training-Free Image-to-3D Multi-Instance Generation with Spatial Fidelitycited by 0
  29. 2026RiGS: Rigid-aware 4D Gaussian Splatting from a Single Monocular Videocited by 0
  30. 2026Inclusive Interactive Collisions for Multi-View Consistent Compositional 3D Generationcited by 0
  31. 2026HIVE-3D: Hierarchical Voxel Enhancement for High-Quality 3D Scene Generationcited by 0
  32. 2026SnapPhysics: A Physics-Aware Scene Graph from a Single View for Interactive Mixed Reality Scenescited by 0
  33. 2026Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstructioncited by 0