From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence
Not on the map. Why?
Abstract (arXiv)
Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains. Consequently, the notion of applying learning methods to a particular problem set has become an established and valuable modus operandi to advance a particular field. In this article we argue that such an approach does not straightforwardly extended to robotics -- or to embodied intelligence more generally: systems which engage in a purposeful exchange of energy and information with a physical environment. In particular, the purview of embodied intelligent agents extends significantly beyond the typical considerations of main-stream machine learning approaches, which typically (i) do not consider operation under conditions significantly different from those encountered during training; (ii) do not consider the often substantial, long-lasting and potentially safety-critical nature of interactions during learning and deployment; (iii) do not require ready adaptation to novel tasks while at the same time (iv) effectively and efficiently curating and extending their models of the world through targeted and deliberate actions. In reality, therefore, these limitations result in learning-based systems which suffer from many of the same operational shortcomings as more traditional, engineering-based approaches when deployed on a robot outside a well defined, and often narrow operating envelope. Contrary to viewing embodied intelligence as another application domain for machine learning, here we argue that it is in fact a key driver for the advancement of machine learning technology. In this article our goal is to highlight challenges and opportunities that are specific to embodied intelligence and to propose research directions which may significantly advance the state-of-the-art in robot learning.
Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 91 of its 191 references that are in the map. First posted on arXiv 28 Oct 2021.
Builds on
91 of 191 references.
References · 91
- 2017Attention Is All You Needcited by 60,361
- 2013Intriguing properties of neural networkscited by 5,155
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
- 2018World Modelscited by 1,513
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
38 more
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
- 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
- 2018Learning Dexterous In-Hand Manipulationcited by 687
- 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
- 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
- 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 392
- 2018GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoderscited by 383
- 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
- 2018Towards a Definition of Disentangled Representationscited by 296
- 2020Integrated Task and Motion Planningcited by 290
- 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 281
- 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
- 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
- 2016Variational Intrinsic Controlcited by 267
- 2019Compositionality decomposed: how do neural networks generalise?cited by 265
- 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 252
- 2019Learning Discrete Structures for Graph Neural Networkscited by 231
- 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
- 2019Recurrent Independent Mechanismscited by 219
- 2017Learning Explanatory Rules from Noisy Datacited by 211
- 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
- 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
- 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
- 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 176
- 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
- 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
- 2016Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Datacited by 156
- 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 155
- 2019Continual Unsupervised Representation Learningcited by 147
- 2019Neural Logic Machinescited by 145
- 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
- 2020Physically Realizable Adversarial Examples for LiDAR Object Detectioncited by 137
- 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
- 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
- 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
50 most cited of 91 All 91 (JSON)
Built on by
Citing papers per year: 2021: 1; 2022: 3; 2023: 8; 2024: 9; 2025: 9; 2026: 5.
35 papers.
Sort:
By area:
- 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 96
- 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
- 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
- 2023Advances in Embodied Navigation Using Large Language Models: A Surveycited by 14
- 2024From Efficient Multimodal Models to World Models: A Surveycited by 13
- 2022Tree Detection and Diameter Estimation Based on Deep Learningcited by 12
- 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
- 2025CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videoscited by 9
- 2023WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Modelcited by 6
- 2024Language-Driven Visual Consensus for Zero-Shot Semantic Segmentationcited by 6
- 2024Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Modelscited by 6
- 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distancecited by 6
23 more
- 2026Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuningcited by 6
- 2022The Risks of Machine Learning Systemscited by 5
- 2023Invariance is Key to Generalization: Examining the Role of Representation in Sim-to-Real Transfer for Visual Navigationcited by 4
- 2025TongSIM: A General Platform for Simulating Intelligent Machinescited by 4
- 2021Social Neuro AI: Social Interaction as the "dark matter" of AIcited by 3
- 2022Provably Efficient Lifelong Reinforcement Learning with Linear Function Approximationcited by 2
- 2023Learning Type-Generalized Actions for Symbolic Planningcited by 2
- 2023Compositional Learning of Visually-Grounded Concepts Using Reinforcementcited by 2
- 2025Ark: An Open-source Python-based Framework for Robot Learningcited by 2
- 2025Multimodal Data Storage and Retrieval for Embodied AI: A Surveycited by 2
- 2025DINOv3-Diffusion Policy: Self-Supervised Large Visual Model for Visuomotor Diffusion Policy Learningcited by 2
- 2023Enactive Artificial Intelligence: Subverting Gender Norms in Robot-Human Interactioncited by 1
- 2023Scaling Distributed Multi-task Reinforcement Learning with Experience Sharingcited by 1
- 2024Sparse and Transferable Universal Singular Vectors Attackcited by 1
- 2024Bidirectional Progressive Neural Networks with Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfercited by 1
- 2025Embodied Intelligence: The Key to Unblocking Generalized Artificial Intelligencecited by 1
- 2026FlowHijack: A Dynamics-Aware Backdoor Attack on Flow-Matching Vision-Language-Action Modelscited by 1
- 2026VEOcc: Voxel-Centric Online Semantic Occupancy Prediction For Embodied Scene Understandingcited by 1
- 2024OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learningcited by 0
- 2024Generative AI Systems: A Systems-based Perspective on Generative AIcited by 0
- 2025Situating AI Agents in their World: Aspective Agentic AI for Dynamic Partially Observable Information Systemscited by 0
- 2026MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agentscited by 0
- 2026Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Taskscited by 0
- 2026Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Tasks1 Oct 2026
- 2026MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agents26 Jul 2026
- 2026VEOcc: Voxel-Centric Online Semantic Occupancy Prediction For Embodied Scene Understanding24 May 2026
- 2026FlowHijack: A Dynamics-Aware Backdoor Attack on Flow-Matching Vision-Language-Action Models30 Mar 2026
- 2026Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuning6 Jan 2026
- 2025TongSIM: A General Platform for Simulating Intelligent Machines23 Dec 2025
- 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distance30 Sep 2025
- 2025DINOv3-Diffusion Policy: Self-Supervised Large Visual Model for Visuomotor Diffusion Policy Learning22 Sep 2025
- 2025Situating AI Agents in their World: Aspective Agentic AI for Dynamic Partially Observable Information Systems3 Sep 2025
- 2025Multimodal Data Storage and Retrieval for Embodied AI: A Survey19 Aug 2025
- 2025CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videos22 Jul 2025
- 2025Ark: An Open-source Python-based Framework for Robot Learning24 Jun 2025
23 more
- 2025Embodied Intelligence: The Key to Unblocking Generalized Artificial Intelligence11 May 2025
- 2025Multi-agent Embodied AI: Advances and Future Directions8 May 2025
- 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulation21 Aug 2024
- 2024Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models15 Aug 2024
- 2024From Efficient Multimodal Models to World Models: A Survey27 Jun 2024
- 2024Generative AI Systems: A Systems-based Perspective on Generative AI25 Jun 2024
- 2024Language-Driven Visual Consensus for Zero-Shot Semantic Segmentation13 Mar 2024
- 2024Bidirectional Progressive Neural Networks with Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfer6 Mar 2024
- 2024Vision-Language Navigation with Embodied Intelligence: A Survey22 Feb 2024
- 2024Sparse and Transferable Universal Singular Vectors Attack25 Jan 2024
- 2024OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning22 Jan 2024
- 2023Levels of AGI for Operationalizing Progress on the Path to AGI4 Nov 2023
- 2023Advances in Embodied Navigation Using Large Language Models: A Survey1 Nov 2023
- 2023Invariance is Key to Generalization: Examining the Role of Representation in Sim-to-Real Transfer for Visual Navigation23 Oct 2023
- 2023Compositional Learning of Visually-Grounded Concepts Using Reinforcement8 Sep 2023
- 2023WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Model30 Aug 2023
- 2023Learning Type-Generalized Actions for Symbolic Planning9 Aug 2023
- 2023Scaling Distributed Multi-task Reinforcement Learning with Experience Sharing11 Jul 2023
- 2023Enactive Artificial Intelligence: Subverting Gender Norms in Robot-Human Interaction17 Jan 2023
- 2022Tree Detection and Diameter Estimation Based on Deep Learning31 Oct 2022
- 2022Provably Efficient Lifelong Reinforcement Learning with Linear Function Approximation1 Jun 2022
- 2022The Risks of Machine Learning Systems21 Apr 2022
- 2021Social Neuro AI: Social Interaction as the "dark matter" of AI31 Dec 2021
- 2024From Efficient Multimodal Models to World Models: A Surveycited by 13
- 2024From Efficient Multimodal Models to World Models: A Survey27 Jun 2024
- 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 96
- 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
- 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
- 2023Advances in Embodied Navigation Using Large Language Models: A Surveycited by 14
- 2022Tree Detection and Diameter Estimation Based on Deep Learningcited by 12
- 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
- 2023WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Modelcited by 6
- 2024Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Modelscited by 6
- 2022The Risks of Machine Learning Systemscited by 5
- 2023Invariance is Key to Generalization: Examining the Role of Representation in Sim-to-Real Transfer for Visual Navigationcited by 4
- 2021Social Neuro AI: Social Interaction as the "dark matter" of AIcited by 3
- 2022Provably Efficient Lifelong Reinforcement Learning with Linear Function Approximationcited by 2
12 most cited of 23 All 23 (JSON)
- 2026VEOcc: Voxel-Centric Online Semantic Occupancy Prediction For Embodied Scene Understanding24 May 2026
- 2025Multimodal Data Storage and Retrieval for Embodied AI: A Survey19 Aug 2025
- 2025Ark: An Open-source Python-based Framework for Robot Learning24 Jun 2025
- 2025Embodied Intelligence: The Key to Unblocking Generalized Artificial Intelligence11 May 2025
- 2025Multi-agent Embodied AI: Advances and Future Directions8 May 2025
- 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulation21 Aug 2024
- 2024Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Models15 Aug 2024
- 2024Generative AI Systems: A Systems-based Perspective on Generative AI25 Jun 2024
- 2024Vision-Language Navigation with Embodied Intelligence: A Survey22 Feb 2024
- 2024Sparse and Transferable Universal Singular Vectors Attack25 Jan 2024
- 2024OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learning22 Jan 2024
- 2023Levels of AGI for Operationalizing Progress on the Path to AGI4 Nov 2023
12 newest of 23 All 23 (JSON)