# REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment

arXiv: https://arxiv.org/abs/2405.18525 · HTML: https://mlsanity.com/paper/2405.18525/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 70 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 30 of 61)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model](https://mlsanity.com/paper/2310.15110/) (2023)
- [Zero-Shot Text-Guided Object Generation with Dream Fields](https://mlsanity.com/paper/2112.01455/) (2021)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [LangSplat: 3D Language Gaussian Splatting](https://mlsanity.com/paper/2312.16084/) (2023)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields](https://mlsanity.com/paper/2112.05139/) (2021)
- [Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior](https://mlsanity.com/paper/2303.14184/) (2023)

## Built on by (most cited first, 30 of 45)

- [MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation](https://mlsanity.com/paper/2412.03558/) (2024)
- [PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers](https://mlsanity.com/paper/2506.05573/) (2025)
- [LangSplatV2: High-dimensional 3D Language Gaussian Splatting with 450+ FPS](https://mlsanity.com/paper/2507.07136/) (2025)
- [DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusion](https://mlsanity.com/paper/2507.22825/) (2025)
- [Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint](https://mlsanity.com/paper/2410.15391/) (2024)
- [SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model](https://mlsanity.com/paper/2512.10957/) (2025)
- [MesaTask: Towards Task-Driven Tabletop Scene Generation via 3D Spatial Reasoning](https://mlsanity.com/paper/2509.22281/) (2025)
- [PAT3D: Physics-Augmented Text-to-3D Scene Generation](https://mlsanity.com/paper/2511.21978/) (2025)
- [DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation](https://mlsanity.com/paper/2505.18078/) (2025)
- [LAYOUTDREAMER: Physics-guided Layout for Text-to-3D Compositional Scene Generation](https://mlsanity.com/paper/2502.01949/) (2025)
- [3D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Image](https://mlsanity.com/paper/2604.04406/) (2026)
- [CompGS: Unleashing 2D Compositionality for Compositional Text-to-3D via Dynamically Optimizing 3D Gaussians](https://mlsanity.com/paper/2410.20723/) (2024)
- [3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework](https://mlsanity.com/paper/2512.17459/) (2025)
- [Semantic Score Distillation Sampling for Compositional Text-to-3D Generation](https://mlsanity.com/paper/2410.09009/) (2024)
- [AToM: Aligning Text-to-Motion Model at Event-Level with GPT-4Vision Reward](https://mlsanity.com/paper/2411.18654/) (2024)
- [Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimization](https://mlsanity.com/paper/2507.14841/) (2025)
- [GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence](https://mlsanity.com/paper/2511.21945/) (2025)
- [I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learners](https://mlsanity.com/paper/2512.13683/) (2025)
- [SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generation](https://mlsanity.com/paper/2602.22785/) (2026)
- [One Video, One World: Turning Monocular Video into Physical 4D Scenes](https://mlsanity.com/paper/2606.31388/) (2026)
- [Flash Sculptor: Modular 3D Worlds from Objects](https://mlsanity.com/paper/2504.06178/) (2025)
- [ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editing](https://mlsanity.com/paper/2509.23607/) (2025)
- [MoCA: Mixture-of-Components Attention for Scalable Compositional 3D Generation](https://mlsanity.com/paper/2512.07628/) (2025)
- [GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction](https://mlsanity.com/paper/2605.23888/) (2026)
- [SimuScene: Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image](https://mlsanity.com/paper/2606.03994/) (2026)
- [Separate to Collaborate: Dual-Stream Diffusion Model for Coordinated Piano Hand Motion Synthesis](https://mlsanity.com/paper/2504.09885/) (2025)
- [InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animation](https://mlsanity.com/paper/2504.10905/) (2025)
- [TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation](https://mlsanity.com/paper/2512.01204/) (2025)
- [SceneLinker: Compositional 3D Scene Generation via Semantic Scene Graph from RGB Sequences](https://mlsanity.com/paper/2602.02974/) (2026)
- [φ-Scene: Physically Grounded Image-to-3D Scene Reconstruction](https://mlsanity.com/paper/2606.21596/) (2026)

## Abstract (arXiv)

Traditional image-to-3D models often struggle with scenes containing multiple objects due to biases and occlusion complexities. To address this challenge, we present REPARO, a novel approach for compositional 3D asset generation from single images. REPARO employs a two-step process: first, it extracts individual objects from the scene and reconstructs their 3D meshes using off-the-shelf image-to-3D models; then, it optimizes the layout of these meshes through differentiable rendering techniques, ensuring coherent scene composition. By integrating optimal transport-based long-range appearance loss term and high-level semantic loss term in the differentiable rendering, REPARO can effectively recover the layout of 3D assets. The proposed method can significantly enhance object independence, detail accuracy, and overall scene coherence. Extensive evaluation of multi-object scenes demonstrates that our REPARO offers a comprehensive approach to address the complexities of multi-object 3D scene generation from single images.