mlsanityData to 7 Oct 2026

BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments

arXiv 2108.03332 ↗First posted 6 Aug 2021cs.RO cs.AI cs.CVcited by 161

Not on the map. Why?

Cited by 22 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce BEHAVIOR, a benchmark for embodied AI with 100 activities in simulation, spanning a range of everyday household chores such as cleaning, maintenance, and food preparation. These activities are designed to be realistic, diverse, and complex, aiming to reproduce the challenges that agents must face in the real world. Building such a benchmark poses three fundamental difficulties for each activity: definition (it can differ by time, place, or person), instantiation in a simulator, and evaluation. BEHAVIOR addresses these with three innovations. First, we propose an object-centric, predicate logic-based description language for expressing an activity's initial and goal conditions, enabling generation of diverse instances for any activity. Second, we identify the simulator-agnostic features required by an underlying environment to support BEHAVIOR, and demonstrate its realization in one such simulator. Third, we introduce a set of metrics to measure task progress and efficiency, absolute and relative to human demonstrators. We include 500 human demonstrations in virtual reality (VR) to serve as the human ground truth. Our experiments demonstrate that even state of the art embodied AI solutions struggle with the level of realism, diversity, and complexity imposed by the activities in our benchmark. We make BEHAVIOR publicly available at behavior.stanford.edu to facilitate and calibrate the development of new embodied AI solutions.

Key facts. Cited by 161 papers in the map (data through 7 Oct 2026). Builds on 54 of its 98 references that are in the map. First posted on arXiv 6 Aug 2021.

Builds on

54 of 98 references.

References · 54

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  5. 2016OpenAI Gymcited by 1,684
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  9. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  10. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  11. 2018DeepMind Control Suitecited by 935
  12. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
38 more
  1. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  2. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  3. 2019Solving Rubik's Cube with a Robot Handcited by 699
  4. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  5. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  7. 2018Learning Dexterous In-Hand Manipulationcited by 643
  8. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  9. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 484
  10. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  11. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  12. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  13. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  14. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  15. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  16. 2017Embodied Question Answeringcited by 335
  17. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  18. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  19. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  20. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  21. 2020Rearrangement: A Challenge for Embodied AIcited by 153
  22. 2020SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulationcited by 118
  23. 2018Actor and Observer: Joint Modeling of First and Third-Person Videoscited by 102
  24. 2021Visual Room Rearrangementcited by 101
  25. 2018Visual Representations for Semantic Target Driven Navigationcited by 89
  26. 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 89
  27. 2016Interactive Perception: Leveraging Action in Perception and Perception in Actioncited by 77
  28. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 76
  29. 2019Multi-Target Embodied Question Answeringcited by 74
  30. 2019IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Taskscited by 68
  31. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  32. 2021The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AIcited by 64
  33. 2018Neural Modular Control for Embodied Question Answeringcited by 56
  34. 2019Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Taskscited by 56
  35. 2019COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Explorationcited by 55
  36. 2019Deep Visual MPC-Policy Learning for Navigationcited by 54
  37. 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 52
  38. 2019Combining Optimal Control and Learning for Visual Navigation in Novel Environmentscited by 50

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 24; 2023: 29; 2024: 33; 2025: 42; 2026: 33.

161 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,537
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  4. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  5. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 484
  6. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  7. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  8. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 369
  9. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 313
  10. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  11. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  12. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 219
38 more
  1. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  2. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
  3. 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
  4. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 140
  5. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  6. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  7. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  8. 2024A Survey on Large Language Model-Based Game Agentscited by 106
  9. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 101
  10. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 101
  11. 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
  12. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 68
  13. 2024Scaling Instructable Agents Across Many Simulated Worldscited by 67
  14. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 63
  15. 2023Gen2Sim: Scaling up Robot Learning in Simulation with Generative Modelscited by 62
  16. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  17. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 55
  18. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 55
  19. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priorscited by 53
  20. 2022OPD: Single-view 3D Openable Part Detectioncited by 52
  21. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 47
  22. 2022Retrospectives on the Embodied AI Workshopcited by 46
  23. 2023Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Predictioncited by 39
  24. 2025Diffusion Models for Robotic Manipulation: A Surveycited by 39
  25. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  26. 2023Learning Human-to-Robot Handovers from Point Cloudscited by 34
  27. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 30
  28. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  29. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  30. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  31. 2025VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Modelscited by 28
  32. 2025Real-to-Sim Robot Policy Evaluation with Gaussian Splatting Simulation of Soft-Body Interactionscited by 27
  33. 2024Odyssey: Empowering Minecraft Agents with Open-World Skillscited by 26
  34. 2022Last-Mile Embodied Visual Navigationcited by 25
  35. 2024An Interactive Agent Foundation Modelcited by 25
  36. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 24
  37. 2023NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistantscited by 23
  38. 2023Mini-BEHAVIOR: A Procedurally Generated Benchmark for Long-horizon Decision-Making in Embodied AIcited by 23

50 most cited of 161 All 161 (JSON)