# Rearrange Indoor Scenes for Human-Robot Co-Activity

arXiv: https://arxiv.org/abs/2303.05676 · HTML: https://mlsanity.com/paper/2303.05676/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 10 Mar 2023.

## Builds on (most cited first, 16 of 16)

- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Models](https://mlsanity.com/paper/1811.12463/) (2018)
- [Human-centric Indoor Scene Synthesis Using Stochastic Grammar](https://mlsanity.com/paper/1808.08473/) (2018)
- [Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsense](https://mlsanity.com/paper/1909.01507/) (2019)
- [Deep Generative Modeling for Scene Synthesis via Hybrid Representations](https://mlsanity.com/paper/1808.02084/) (2018)
- [Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense](https://mlsanity.com/paper/2004.09044/) (2020)
- [Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars](https://mlsanity.com/paper/1704.00112/) (2017)
- [Sequential Manipulation Planning on Scene Graph](https://mlsanity.com/paper/2207.04364/) (2022)
- [Where to relocate?: Object rearrangement inside cluttered and confined environments for robotic manipulation](https://mlsanity.com/paper/2003.10863/) (2020)
- [Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments](https://mlsanity.com/paper/2103.16095/) (2021)
- [SceneGen: Generative Contextual Scene Augmentation using Scene Graph Priors](https://mlsanity.com/paper/2009.12395/) (2020)
- [Efficient Task Planning for Mobile Manipulation: a Virtual Kinematic Chain Perspective](https://mlsanity.com/paper/2108.01259/) (2021)
- [Designing Human-Robot Coexistence Space](https://mlsanity.com/paper/2011.07374/) (2020)
- [Optimizing Hospital Room Layout to Reduce the Risk of Patient Falls](https://mlsanity.com/paper/2101.03210/) (2021)
- [Towards Mobile Multi-Task Manipulation in a Confined and Integrated Environment with Irregular Objects](https://mlsanity.com/paper/2003.01776/) (2020)

## Built on by (most cited first, 5 of 5)

- [PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI](https://mlsanity.com/paper/2404.09465/) (2024)
- [UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI](https://mlsanity.com/paper/2412.20977/) (2024)
- [Situat3DChange: Situated 3D Change Understanding Dataset for Multimodal Large Language Model](https://mlsanity.com/paper/2510.11509/) (2025)
- [iARCS: Iterative Agentic RL for Controllable 3D Scene Generation](https://mlsanity.com/paper/2608.06161/) (2026)
- [Improving Human-Robot Collaboration via Computational Design](https://mlsanity.com/paper/2303.11425/) (2023)

## Abstract (arXiv)

We present an optimization-based framework for rearranging indoor furniture to accommodate human-robot co-activities better. The rearrangement aims to afford sufficient accessible space for robot activities without compromising everyday human activities. To retain human activities, our algorithm preserves the functional relations among furniture by integrating spatial and semantic co-occurrence extracted from SUNCG and ConceptNet, respectively. By defining the robot's accessible space by the amount of open space it can traverse and the number of objects it can reach, we formulate the rearrangement for human-robot co-activity as an optimization problem, solved by adaptive simulated annealing (ASA) and covariance matrix adaptation evolution strategy (CMA-ES). Our experiments on the SUNCG dataset quantitatively show that rearranged scenes provide an average of 14% more accessible space and 30% more objects to interact with. The quality of the rearranged scenes is qualitatively validated by a human study, indicating the efficacy of the proposed strategy.