mlsanityData to 14 Oct 2026

A New Era in Human Factors Engineering: A Survey of the Applications and Prospects of Large Multimodal Models

arXiv 2405.13426 ↗First posted 22 May 2024cs.HC cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

In recent years, the potential applications of Large Multimodal Models (LMMs) in fields such as healthcare, social psychology, and industrial design have attracted wide research attention, providing new directions for human factors research. For instance, LMM-based smart systems have become novel research subjects of human factors studies, and LMM introduces new research paradigms and methodologies to this field. Therefore, this paper aims to explore the applications, challenges, and future prospects of LMM in the domain of human factors and ergonomics through an expert-LMM collaborated literature review. Specifically, a novel literature review method is proposed, and research studies of LMM-based accident analysis, human modelling and intervention design are introduced. Subsequently, the paper discusses future trends of the research paradigm and challenges of human factors and ergonomics studies in the era of LMMs. It is expected that this study can provide a valuable perspective and serve as a reference for integrating human factors with artificial intelligence.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 24 of its 86 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

24 of 86 references.

References · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  6. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  7. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 333
  8. 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 183
  9. 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 167
  10. 2022PromptChainer: Chaining Large Language Model Prompts through Visual Programmingcited by 134
  11. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  12. 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
12 more
  1. 2023Psy-LLM: Scaling up Global Mental Health Psychological Services with AI-based Large Language Modelscited by 36
  2. 2023Large Language Models as Zero-Shot Human Models for Human-Robot Interactioncited by 30
  3. 2024EEG-GPT: Exploring Capabilities of Large Language Models for EEG Classification and Interpretationcited by 27
  4. 2024LaMI: Large Language Models for Multi-Modal Human-Robot Interactioncited by 21
  5. 2023AccidentGPT: Accident Analysis and Prevention from V2X Environmental Perception with Multi-modal Large Modelcited by 17
  6. 2023Large Language Models in Analyzing Crash Narratives -- A Comparative Study of ChatGPT, BARD and GPT-4cited by 16
  7. 2021A Compact and Interpretable Convolutional Neural Network for Cross-Subject Driver Drowsiness Detection from Single-Channel EEGcited by 12
  8. 2024AccidentGPT: Large Multi-Modal Foundation Model for Traffic Accident Analysiscited by 10
  9. 2024Enhancing Traffic Incident Management with Large Language Models: A Hybrid Machine Learning Approach for Severity Classificationcited by 7
  10. 2023Generative AI for Product Design: Getting the Right Design and the Design Rightcited by 6
  11. 2024Reading Users' Minds from What They Say: An Investigation into LLM-based Empathic Mental Inferencecited by 3
  12. 2024Virtual Co-Pilot: Multimodal Large Language Model-enabled Quick-access Procedures for Single Pilot Operationscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 3.

5 papers.

  1. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
  2. 2026DiLLS: Interactive Diagnosis of LLM-based Multi-agent Systems via Layered Summary of Agent Behaviorscited by 7
  3. 2026Personalize Your Large Vision-language Models With In-context Prompt Tuningcited by 3
  4. 2025A Hybrid Swarm Intelligence Approach for Optimizing Multimodal Large Language Models Deployment in Edge-Cloud-based Federated Learning Environmentscited by 1
  5. 2026InconLens: Interactive Visual Diagnosis of Behavioral Inconsistencies in LLM-based Agentic Systemscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Embracing Large Language and Multimodal Models for Prosthetic Technologies
  2. 2026Large Multimodal Models for Embodied Intelligent Driving: The Next Frontier in Self-Driving?
  3. 2025Artificial Behavior Intelligence: Technology, Challenges, and Future Directions
  4. 2024Leveraging Large Language Models in Human-Robot Interaction: A Critical Analysis of Potential and Pitfalls
  5. 2024Human-Centered Automation
  6. 2025Assessing the Human-Likeness of LLM-Driven Digital Twins in Simulating Health Care System Trust
  7. 2025Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Models
  8. 2026Evaluating Multimodal LLMs across Text and Audio Modalities for Accessible Disaster Assistance