mlsanityData to 14 Oct 2026

Evaluating Multimodal Large Language Models with Daily Composite Tasks in Home Environments

arXiv 2509.17425 ↗First posted 22 Sep 2025cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

A key feature differentiating artificial general intelligence (AGI) from traditional AI is that AGI can perform composite tasks that require a wide range of capabilities. Although embodied agents powered by multimodal large language models (MLLMs) offer rich perceptual and interactive capabilities, it remains largely unexplored whether they can solve composite tasks. In the current work, we designed a set of composite tasks inspired by common daily activities observed in early childhood development. Within a dynamic and simulated home environment, these tasks span three core domains: object understanding, spatial intelligence, and social activity. We evaluated 17 leading proprietary and open-source MLLMs on these tasks. The results consistently showed poor performance across all three domains, indicating a substantial gap between current capabilities and general intelligence requirements. Together, our tasks offer a preliminary framework for evaluating the general capabilities of embodied agents, marking an early but significant step toward the development of embodied MLLMs and their real-world deployment.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 57 references that are in the map. First posted on arXiv 22 Sep 2025.

Builds on

20 of 57 references.

Closest ancestors · 5

Same direction, newest first

  1. 202511Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis27 Aug
  2. 2025MindCube: Spatial Mental Modeling from Limited Views26 Jun
  3. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigation4 Mar
  4. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents13 Feb
  5. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents21 Jan

Other references · 15

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  3. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  4. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  5. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 250
  6. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  7. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  8. 2019On the Measure of Intelligencecited by 144
  9. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  10. 2025InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulationcited by 72
  11. 2024GRUtopia: Dream General Robots in a City at Scalecited by 59
  12. 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 51
3 more
  1. 2019VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learningcited by 29
  2. 2025Virtual Community: An Open World for Humans, Robots, and Societycited by 13
  3. 2025FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Personalizing Embodied Multimodal Large Language Model Agents over Long-term User Interactionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025On Path to Multimodal Generalist: General-Level and General-Bench
  2. 2024From Multimodal LLMs to Generalist Embodied Agents: Methods and Lessons
  3. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches
  4. 2023Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models
  5. 2026Towards general embodied intelligence: integrating large language models, knowledge bases, and reasoning capabilities to build the next generation of AI agents
  6. 2025A Survey on Agentic Multimodal Large Language Models
  7. 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
  8. 2025BiTAgent: A Task-Aware Modular Framework for Bidirectional Coupling between Multimodal Large Language Models and World Models