mlsanityData to 7 Oct 2026

LEGO-Net: Learning Regular Rearrangements of Objects in Rooms

arXiv 2301.09629 ↗First posted 23 Jan 2023cs.CVcited by 75

Not on the map. Why?

Abstract (arXiv)

Humans universally dislike the task of cleaning up a messy room. If machines were to help us with this task, they must understand human criteria for regular arrangements, such as several types of symmetry, co-linearity or co-circularity, spacing uniformity in linear or circular patterns, and further inter-object relationships that relate to style and functionality. Previous approaches for this task relied on human input to explicitly specify goal state, or synthesized scenes from scratch -- but such methods do not address the rearrangement of existing messy scenes without providing a goal state. In this paper, we present LEGO-Net, a data-driven transformer-based iterative method for LEarning reGular rearrangement of Objects in messy rooms. LEGO-Net is partly inspired by diffusion models -- it starts with an initial messy state and iteratively ''de-noises'' the position and orientation of objects to a regular state while reducing distance traveled. Given randomly perturbed object positions and orientations in an existing dataset of professionally-arranged scenes, our method is trained to recover a regular re-arrangement. Results demonstrate that our method is able to reliably rearrange room scenes and outperform other methods. We additionally propose a metric for evaluating regularity in room arrangements using number-theoretic machinery.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Builds on 46 of its 83 references that are in the map. First posted on arXiv 23 Jan 2023.

Builds on

46 of 83 references.

References · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  12. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
34 more
  1. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  2. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  3. 2018Demystifying MMD GANscited by 1,268
  4. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  6. 2016Colorful Image Colorizationcited by 1,113
  7. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  8. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  9. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  10. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  11. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  12. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  13. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  14. 2017SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentationcited by 168
  15. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 154
  16. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  17. 2018GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloudcited by 145
  18. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  19. 2021Visual Room Rearrangementcited by 101
  20. 2018Human-centric Indoor Scene Synthesis Using Stochastic Grammarcited by 73
  21. 2018Deep Generative Modeling for Scene Synthesis via Hybrid Representationscited by 63
  22. 2020Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generationcited by 58
  23. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  24. 2019SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentationcited by 55
  25. 2020End-to-End Optimization of Scene Layoutcited by 52
  26. 2020Generative Layout Modeling using Constraint Graphscited by 46
  27. 2018GRAINS: Generative Recursive Autoencoders for INdoor Scenescited by 38
  28. 2022ConDor: Self-Supervised Canonicalization of 3D Pose for Partial Shapescited by 31
  29. 2019Mechanical Search: Multi-Step Retrieval of a Target Object Occluded by Cluttercited by 30
  30. 2019Monte-Carlo Tree Search for Efficient Visually Guided Rearrangement Planningcited by 23
  31. 2018Fast and Scalable Position-Based Layout Synthesiscited by 15
  32. 2021IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenescited by 9
  33. 2020Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extractioncited by 7
  34. 2020Synchronized Multi-Arm Rearrangement Guided by Mode Graphs with Capacity Constraintscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 21; 2025: 22; 2026: 17.

75 papers.

Sort:

By area:

  1. 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
  2. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesiscited by 159
  3. 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 125
  4. 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
  5. 2024I-Design: Personalized LLM Interior Designercited by 87
  6. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  7. 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 66
  8. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
  9. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  10. 2025SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agentcited by 53
  11. 2023UGG: Unified Generative Graspingcited by 30
  12. 2024Mixed Diffusion for 3D Indoor Scene Synthesiscited by 30
38 more
  1. 2023Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangementcited by 29
  2. 2023NAP: Neural 3D Articulation Priorcited by 25
  3. 2024CityCraft: A Real Crafter for 3D City Generationcited by 25
  4. 2023SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphscited by 24
  5. 2024SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generationcited by 24
  6. 2025FirePlace: Geometric Refinements of LLM Common Sense Reasoning for 3D Object Placementcited by 24
  7. 2024Chat2Layout: Interactive 3D Furniture Layout with a Multimodal LLMcited by 20
  8. 2024DeBaRA: Denoising-Based 3D Room Arrangement Generationcited by 20
  9. 2023Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariancecited by 19
  10. 2025CasaGPT: Cuboid Arrangement and Scene Assembly for Interior Designcited by 14
  11. 2025OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimizationcited by 14
  12. 2024Forest2Seq: Revitalizing Order Prior for Sequential Indoor Scene Synthesiscited by 12
  13. 2024FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenescited by 10
  14. 2024VRCopilot: Authoring 3D Layouts with Generative AI Models in VRcited by 10
  15. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  16. 2025ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latentscited by 10
  17. 2024"Set It Up!": Functional Object Arrangement with Compositional Generative Modelscited by 9
  18. 2025Steerable Scene Generation with Post Training and Inference-Time Searchcited by 8
  19. 2025PlaceIt3D: Language-Guided Object Placement in Real 3D Scenescited by 8
  20. 2025FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Promptscited by 8
  21. 2023Structural Concept Learning via Graph Attention for Multi-Level Rearrangement Planningcited by 7
  22. 2023LVDiffusor: Distilling Functional Rearrangement Priors from Large Models into Diffusorcited by 7
  23. 2024Lay-A-Scene: Personalized 3D Object Arrangement Using Text-to-Image Priorscited by 7
  24. 2025LACONIC: A 3D Layout Adapter for Controllable Image Creationcited by 6
  25. 2025SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesiscited by 6
  26. 2024Online Diffusion-Based 3D Occupancy Prediction at the Frontier with Probabilistic Map Reconciliationcited by 5
  27. 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editingcited by 5
  28. 2023Blocks2World: Controlling Realistic Scenes with Editable Primitivescited by 4
  29. 2023SceneScore: Learning a Cost Function for Object Arrangementcited by 4
  30. 2025Learning 3D Scene Analogies with Neural Contextual Scene Mapscited by 4
  31. 2025MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detectioncited by 4
  32. 2025ScanEdit: Hierarchically-Guided Functional 3D Scan Editingcited by 4
  33. 2024Grounded Compositional and Diverse Text-to-3D with Pretrained Multi-View Diffusion Modelcited by 3
  34. 2024Exploring the Impact of Synthetic Data for Aerial-view Human Detectioncited by 3
  35. 2025BOOTPLACE: Bootstrapped Object Placement with Detection Transformerscited by 3
  36. 2025Computer-Aided Layout Generation for Building Design: A Reviewcited by 3
  37. 20253D Software Synthesis Driven by Constraint-Expressive Intermediate Representationcited by 3
  38. 2025VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangementcited by 3

50 most cited of 75 All 75 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 20263D Scene Graph Prediction: Generating Hierarchical Models from Partially Observed Environments
  2. 2026FlowScene: Style-Consistent Indoor Scene Generation with Multimodal Graph Rectified Flow
  3. 2023"Tidy Up the Table": Grounding Common-sense Objective for Tabletop Object Rearrangement
  4. 2026Pair2Scene: Learning Local Object Relations for Procedural Scene Generation
  5. 2024MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes
  6. 2024Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Plane
  7. 2025gen2seg: Generative Models Enable Generalizable Instance Segmentation
  8. 2025RoomCraft: Controllable and Complete 3D Indoor Scene Generation