# HoME: a Household Multimodal Environment

arXiv: https://arxiv.org/abs/1711.11017 · HTML: https://mlsanity.com/paper/1711.11017/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2017.

## Built on by (most cited first, 20 of 20)

- [Building Cooperative Embodied Agents Modularly with Large Language Models](https://mlsanity.com/paper/2307.02485/) (2023)
- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Learning to Understand Goal Specifications by Modelling Reward](https://mlsanity.com/paper/1806.01946/) (2018)
- [Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention](https://mlsanity.com/paper/1812.04155/) (2018)
- [CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents](https://mlsanity.com/paper/2110.10067/) (2021)
- [LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments](https://mlsanity.com/paper/2407.10031/) (2024)
- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [Core Challenges in Embodied Vision-Language Planning](https://mlsanity.com/paper/2106.13948/) (2021)
- [Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production](https://mlsanity.com/paper/2301.12688/) (2023)
- [Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents](https://mlsanity.com/paper/2110.05769/) (2021)
- [Collaborative Visual Navigation](https://mlsanity.com/paper/2107.01151/) (2021)
- [LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planning](https://mlsanity.com/paper/2502.05453/) (2025)
- [Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene](https://mlsanity.com/paper/2108.02846/) (2021)
- [Advancing Drug Discovery with Enhanced Chemical Understanding via Asymmetric Contrastive Multimodal Learning](https://mlsanity.com/paper/2311.06456/) (2023)
- [Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method](https://mlsanity.com/paper/2111.14267/) (2021)
- [UnrealROX+: An Improved Tool for Acquiring Synthetic Data from Virtual 3D Environments](https://mlsanity.com/paper/2104.11776/) (2021)
- [VECA : A Toolkit for Building Virtual Environments to Train and Test Human-like Agents](https://mlsanity.com/paper/2105.00762/) (2021)
- [Foundations of Multisensory Artificial Intelligence](https://mlsanity.com/paper/2404.18976/) (2024)
- [Core Challenges in Embodied Vision-Language Planning](https://mlsanity.com/paper/2304.02738/) (2023)
- [VR-based generation of photorealistic synthetic data for training hand-object tracking models](https://mlsanity.com/paper/2401.17874/) (2024)

## Abstract (arXiv)

We introduce HoME: a Household Multimodal Environment for artificial agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context. HoME integrates over 45,000 diverse 3D house layouts based on the SUNCG dataset, a scale which may facilitate learning, generalization, and transfer. HoME is an open-source, OpenAI Gym-compatible platform extensible to tasks in reinforcement learning, language grounding, sound-based navigation, robotics, multi-agent learning, and more. We hope HoME better enables artificial agents to learn as humans do: in an interactive, multimodal, and richly contextualized setting.