Human-Centric Intelligence in the Era of Foundation Models: A Survey
Not on the map. Why?
Abstract (arXiv)
Human-centric intelligence is evolving in the foundation-model era, with growing emphasis on scale, transferability, and general-purpose modeling. Yet it has not fully integrated with foundation models to achieve the comparable progress seen in them. More importantly, recent advances across this broad landscape remain fragmented across tasks, modalities, and research communities, leaving their intrinsic conceptual and methodological connections unclear. To bridge these divides and rethink human-centric intelligence in the foundation-model era, we introduce a full-spectrum human context taxonomy that integrates six interconnected levels by viewing humans as observable subjects through visual appearance and spatial geometry, as dynamic actors through kinematic dynamics and interaction modeling, and as situated agents through world simulation and embodied agency. We next present the methodological foundations of the field, covering human-centric data families, computational architecture paradigms, and representative training and inference optimization strategies. We then systematically review representative methods across these levels and organize the associated datasets, benchmarks, and evaluation metrics. We further discuss open challenges and promising research directions toward human-centric intelligence that is scalable, trustworthy, physically grounded, and deployable, aiming to provide a coherent framework and practical reference for advancing the field. Finally, we provide a systematically organized and continuously updated collection of human-centric AI literature and resources on our project page.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 160 of its 454 references that are in the map. First posted on arXiv 18 Aug 2026.
Builds on
160 of 454 references.
References · 160
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
- 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 135
- 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 128
- 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
- 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 101
- 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Datacited by 94
- 2025GMT: General Motion Tracking for Humanoid Whole-Body Controlcited by 83
- 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 83
- 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 80
- 2026Being-H0.7: A Latent World-Action Model from Egocentric Videoscited by 75
38 more
- 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 70
- 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 69
- 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
- 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 47
- 2024ViViD: Video Virtual Try-on using Diffusion Modelscited by 42
- 2025Motion Anything: Any to Motion Generationcited by 42
- 2025Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Datasetcited by 40
- 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 39
- 2025BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learningcited by 38
- 2025Human3R: Everyone Everywhere All at Oncecited by 37
- 2026Kimodo: Scaling Controllable Human Motion Generationcited by 37
- 2025WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Controlcited by 36
- 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 36
- 2025Motion-X++: A Large-Scale Multimodal 3D Whole-body Human Motion Datasetcited by 33
- 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 32
- 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videoscited by 31
- 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 30
- 2025RealisDance-DiT: Simple yet Strong Baseline towards Controllable Character Animation in the Wildcited by 29
- 2025Video-STAR: Reinforcing Open-Vocabulary Action Recognition with Toolscited by 26
- 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 26
- 2024Scaling Large Motion Models with Million-Level Human Motionscited by 22
- 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 22
- 2025The Quest for Generalizable Motion Generation: Data, Model, and Evaluationcited by 22
- 2026DreamID-Omni: Unified Framework for Controllable Human-Centric Audio-Video Generationcited by 20
- 2026EgoLive: A Large-Scale Egocentric Dataset from Real-World Human Taskscited by 20
- 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 19
- 2025Embody 3D: A Large-scale Multimodal Motion and Behavior Datasetcited by 19
- 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 18
- 2025From Language to Locomotion: Retargeting-free Humanoid Control via Motion Latent Guidancecited by 18
- 2025DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Frameworkcited by 17
- 2026Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gesturescited by 16
- 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 16
- 2025ChatMotion: A Multimodal Multi-Agent for Human Motion Analysiscited by 14
- 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 14
- 2024Face-MLLM: A Large Face Perception Modelcited by 13
- 2025OpenEgo: A Large-Scale Multimodal Egocentric Dataset for Dexterous Manipulationcited by 13
- 2025KlingAvatar 2.0 Technical Reportcited by 13
- 2025EchoMotion: Unified Human Video and Motion Generation via Dual-Modality Diffusion Transformercited by 13
50 most cited of 160 All 160 (JSON)
Built on by
Not cited yet.