# CHALET: Cornell House Agent Learning Environment

arXiv: https://arxiv.org/abs/1801.07357 · HTML: https://mlsanity.com/paper/1801.07357/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2018.

## Built on by (most cited first, 30 of 38)

- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation](https://mlsanity.com/paper/2203.10421/) (2022)
- [Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation](https://mlsanity.com/paper/2203.02764/) (2022)
- [Language Models Meet World Models: Embodied Experiences Enhance Language Models](https://mlsanity.com/paper/2305.10626/) (2023)
- [Habitat-Matterport 3D Semantics Dataset](https://mlsanity.com/paper/2210.05633/) (2022)
- [Octopus: Embodied Vision-Language Programmer from Environmental Feedback](https://mlsanity.com/paper/2310.08588/) (2023)
- [CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents](https://mlsanity.com/paper/2110.10067/) (2021)
- [HAZARD Challenge: Embodied Decision Making in Dynamically Changing Environments](https://mlsanity.com/paper/2401.12975/) (2024)
- [Core Challenges in Embodied Vision-Language Planning](https://mlsanity.com/paper/2106.13948/) (2021)
- [A Systematic Survey of Text Worlds as Embodied Natural Language Environments](https://mlsanity.com/paper/2107.04132/) (2021)
- [HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers](https://mlsanity.com/paper/2205.09747/) (2022)
- [Unsupervised Kinematic Motion Detection for Part-segmented 3D Shape Collections](https://mlsanity.com/paper/2206.08497/) (2022)
- [Virtual Community: An Open World for Humans, Robots, and Society](https://mlsanity.com/paper/2508.14893/) (2025)
- [LUMINOUS: Indoor Scene Generation for Embodied AI Challenges](https://mlsanity.com/paper/2111.05527/) (2021)
- [LEGENT: Open Platform for Embodied Agents](https://mlsanity.com/paper/2404.18243/) (2024)
- [MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation](https://mlsanity.com/paper/2103.11374/) (2021)
- [CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation](https://mlsanity.com/paper/2306.10322/) (2023)
- [Are We There Yet? Learning to Localize in Embodied Instruction Following](https://mlsanity.com/paper/2101.03431/) (2021)
- [Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scene](https://mlsanity.com/paper/2108.02846/) (2021)
- [A call for embodied AI](https://mlsanity.com/paper/2402.03824/) (2024)
- [Embodied AI-Driven Operation of Smart Cities: A Concise Review](https://mlsanity.com/paper/2108.09823/) (2021)
- [Explore the Potential Performance of Vision-and-Language Navigation Model: a Snapshot Ensemble Method](https://mlsanity.com/paper/2111.14267/) (2021)
- [UnrealROX+: An Improved Tool for Acquiring Synthetic Data from Virtual 3D Environments](https://mlsanity.com/paper/2104.11776/) (2021)
- [Where Do We Go from Here? Multi-scale Allocentric Relational Inference from Natural Spatial Descriptions](https://mlsanity.com/paper/2402.16364/) (2024)
- [Learning to Build by Building Your Own Instructions](https://mlsanity.com/paper/2410.01111/) (2024)
- [Vision Language Models Cannot Plan, but Can They Formalize?](https://mlsanity.com/paper/2509.21576/) (2025)
- [Deep Learning for Embodied Vision Navigation: A Survey](https://mlsanity.com/paper/2108.04097/) (2021)
- [Evaluating Continual Learning Algorithms by Generating 3D Virtual Environments](https://mlsanity.com/paper/2109.07855/) (2021)

## Abstract (arXiv)

We present CHALET, a 3D house simulator with support for navigation and manipulation. CHALET includes 58 rooms and 10 house configuration, and allows to easily create new house and room layouts. CHALET supports a range of common household activities, including moving objects, toggling appliances, and placing objects inside closeable containers. The environment and actions available are designed to create a challenging domain to train and evaluate autonomous agents, including for tasks that combine language, vision, and planning in a dynamic environment.