# Evaluating Multimodal Large Language Models with Daily Composite Tasks in Home Environments

arXiv: https://arxiv.org/abs/2509.17425 · HTML: https://mlsanity.com/paper/2509.17425/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 20 of its 57 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation](https://mlsanity.com/paper/2007.04954/) (2020)
- [NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation](https://mlsanity.com/paper/2402.15852/) (2024)
- [Habitat 3.0: A Co-Habitat for Humans, Avatars and Robots](https://mlsanity.com/paper/2310.13724/) (2023)
- [EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents](https://mlsanity.com/paper/2502.09560/) (2025)
- [On the Measure of Intelligence](https://mlsanity.com/paper/1911.01547/) (2019)
- [MindCube: Spatial Mental Modeling from Limited Views](https://mlsanity.com/paper/2506.21458/) (2025)
- [Large Language Models as Generalizable Policies for Embodied Tasks](https://mlsanity.com/paper/2310.17722/) (2023)
- [InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation](https://mlsanity.com/paper/2507.17520/) (2025)
- [GRUtopia: Dream General Robots in a City at Scale](https://mlsanity.com/paper/2407.10943/) (2024)
- [EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents](https://mlsanity.com/paper/2501.11858/) (2025)
- [Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage](https://mlsanity.com/paper/2412.15606/) (2024)
- [WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation](https://mlsanity.com/paper/2503.02247/) (2025)
- [VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learning](https://mlsanity.com/paper/1903.05757/) (2019)
- [Virtual Community: An Open World for Humans, Robots, and Society](https://mlsanity.com/paper/2508.14893/) (2025)
- [11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis](https://mlsanity.com/paper/2508.20068/) (2025)
- [FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation](https://mlsanity.com/paper/2506.09081/) (2025)

## Built on by (most cited first, 1 of 1)

- [Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactions](https://mlsanity.com/paper/2605.26256/) (2026)

## Abstract (arXiv)

A key feature differentiating artificial general intelligence (AGI) from traditional AI is that AGI can perform composite tasks that require a wide range of capabilities. Although embodied agents powered by multimodal large language models (MLLMs) offer rich perceptual and interactive capabilities, it remains largely unexplored whether they can solve composite tasks. In the current work, we designed a set of composite tasks inspired by common daily activities observed in early childhood development. Within a dynamic and simulated home environment, these tasks span three core domains: object understanding, spatial intelligence, and social activity. We evaluated 17 leading proprietary and open-source MLLMs on these tasks. The results consistently showed poor performance across all three domains, indicating a substantial gap between current capabilities and general intelligence requirements. Together, our tasks offer a preliminary framework for evaluating the general capabilities of embodied agents, marking an early but significant step toward the development of embodied MLLMs and their real-world deployment.