# ClutterGen: A Cluttered Scene Generator for Robot Learning

arXiv: https://arxiv.org/abs/2407.05425 · HTML: https://mlsanity.com/paper/2407.05425/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2024.

## Built on by (most cited first, 8 of 8)

- [Steerable Scene Generation with Post Training and Inference-Time Search](https://mlsanity.com/paper/2505.04831/) (2025)
- [PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement](https://mlsanity.com/paper/2602.14968/) (2026)
- [Neural Fidelity Calibration for Informative Sim-to-Real Adaptation](https://mlsanity.com/paper/2504.08604/) (2025)
- [GOPLA: Generalizable Object Placement Learning via Synthetic Augmentation of Human Arrangement](https://mlsanity.com/paper/2510.14627/) (2025)
- [PhyScene3D: Physically Consistent Interactive 3D Tabletop Scene Generation](https://mlsanity.com/paper/2606.01649/) (2026)
- [GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learning](https://mlsanity.com/paper/2609.05927/) (2026)
- [HetScene: Heterogeneity-Aware Diffusion for Dense Indoor Scene Generation](https://mlsanity.com/paper/2605.13586/) (2026)
- [WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents](https://mlsanity.com/paper/2609.40325/) (2026)

## Abstract (arXiv)

We introduce ClutterGen, a physically compliant simulation scene generator capable of producing highly diverse, cluttered, and stable scenes for robot learning. Generating such scenes is challenging as each object must adhere to physical laws like gravity and collision. As the number of objects increases, finding valid poses becomes more difficult, necessitating significant human engineering effort, which limits the diversity of the scenes. To overcome these challenges, we propose a reinforcement learning method that can be trained with physics-based reward signals provided by the simulator. Our experiments demonstrate that ClutterGen can generate cluttered object layouts with up to ten objects on confined table surfaces. Additionally, our policy design explicitly encourages the diversity of the generated scenes for open-ended generation. Our real-world robot results show that ClutterGen can be directly used for clutter rearrangement and stable placement policy training.