HoME: a Household Multimodal Environment
Not on the map. Why?
Abstract (arXiv)
We introduce HoME: a Household Multimodal Environment for artificial agents to learn from vision, audio, semantics, physics, and interaction with objects and other agents, all within a realistic context. HoME integrates over 45,000 diverse 3D house layouts based on the SUNCG dataset, a scale which may facilitate learning, generalization, and transfer. HoME is an open-source, OpenAI Gym-compatible platform extensible to tasks in reinforcement learning, language grounding, sound-based navigation, robotics, multi-agent learning, and more. We hope HoME better enables artificial agents to learn as humans do: in an interactive, multimodal, and richly contextualized setting.
Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Nov 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2018: 2; 2019: 0; 2020: 0; 2021: 9; 2022: 1; 2023: 4; 2024: 3; 2025: 1.
20 papers.
Sort:
By area:
- 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
- 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
- 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
- 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
- 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 38
- 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environmentscited by 32
- 2021Deep Learning Based 3D Segmentation: A Surveycited by 30
- 2021Core Challenges in Embodied Vision-Language Planningcited by 30
- 2023Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Productioncited by 21
- 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agentscited by 19
- 2021Collaborative Visual Navigationcited by 18
- 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planningcited by 18
8 more
- 2021Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scenecited by 7
- 2023Advancing Drug Discovery with Enhanced Chemical Understanding via Asymmetric Contrastive Multimodal Learningcited by 5
- 2021Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Methodcited by 4
- 2021UnrealROX+: An Improved Tool for Acquiring Synthetic Data from Virtual 3D Environmentscited by 3
- 2021VECA : A Toolkit for Building Virtual Environments to Train and Test Human-like Agentscited by 1
- 2024Foundations of Multisensory Artificial Intelligencecited by 1
- 2023Core Challenges in Embodied Vision-Language Planningcited by 0
- 2024VR-based generation of photorealistic synthetic data for training hand-object tracking modelscited by 0
- 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planning8 Feb 2025
- 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments14 Jul 2024
- 2024Foundations of Multisensory Artificial Intelligence29 Apr 2024
- 2024VR-based generation of photorealistic synthetic data for training hand-object tracking models31 Jan 2024
- 2023Advancing Drug Discovery with Enhanced Chemical Understanding via Asymmetric Contrastive Multimodal Learning11 Nov 2023
- 2023Building Cooperative Embodied Agents Modularly with Large Language Models5 Jul 2023
- 2023Core Challenges in Embodied Vision-Language Planning5 Apr 2023
- 2023Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production30 Jan 2023
- 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions7 Sep 2022
- 2021Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method28 Nov 2021
- 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents19 Oct 2021
- 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents12 Oct 2021
8 more
- 2021Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene5 Aug 2021
- 2021Collaborative Visual Navigation2 Jul 2021
- 2021Core Challenges in Embodied Vision-Language Planning26 Jun 2021
- 2021VECA : A Toolkit for Building Virtual Environments to Train and Test Human-like Agents3 May 2021
- 2021UnrealROX+: An Improved Tool for Acquiring Synthetic Data from Virtual 3D Environments23 Apr 2021
- 2021Deep Learning Based 3D Segmentation: A Survey9 Mar 2021
- 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention10 Dec 2018
- 2018Learning to Understand Goal Specifications by Modelling Reward5 Jun 2018
- 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
- 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environmentscited by 32
- 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planningcited by 18
- 2025LLM-Powered Decentralized Generative Agents with Adaptive Hierarchical Knowledge Graph for Cooperative Planning8 Feb 2025
- 2024LLaMAR: Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments14 Jul 2024
- 2023Building Cooperative Embodied Agents Modularly with Large Language Models5 Jul 2023
- 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
- 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 38
- 2021Deep Learning Based 3D Segmentation: A Surveycited by 30
- 2021Core Challenges in Embodied Vision-Language Planningcited by 30
- 2023Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Productioncited by 21
- 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agentscited by 19
- 2021Collaborative Visual Navigationcited by 18
- 2021Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scenecited by 7
- 2023Advancing Drug Discovery with Enhanced Chemical Understanding via Asymmetric Contrastive Multimodal Learningcited by 5
- 2021Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Methodcited by 4
- 2021UnrealROX+: An Improved Tool for Acquiring Synthetic Data from Virtual 3D Environmentscited by 3
- 2021VECA : A Toolkit for Building Virtual Environments to Train and Test Human-like Agentscited by 1
12 most cited of 15 All 15 (JSON)
- 2024Foundations of Multisensory Artificial Intelligence29 Apr 2024
- 2024VR-based generation of photorealistic synthetic data for training hand-object tracking models31 Jan 2024
- 2023Advancing Drug Discovery with Enhanced Chemical Understanding via Asymmetric Contrastive Multimodal Learning11 Nov 2023
- 2023Core Challenges in Embodied Vision-Language Planning5 Apr 2023
- 2023Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Production30 Jan 2023
- 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions7 Sep 2022
- 2021Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method28 Nov 2021
- 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents19 Oct 2021
- 2021Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents12 Oct 2021
- 2021Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene5 Aug 2021
- 2021Collaborative Visual Navigation2 Jul 2021
- 2021Core Challenges in Embodied Vision-Language Planning26 Jun 2021
12 newest of 15 All 15 (JSON)