mlsanityData to 14 Oct 2026

From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence

arXiv 2110.15245 ↗First posted 28 Oct 2021cs.RO cs.LGcited by 35

Not on the map. Why?

Abstract (arXiv)

Machine learning has long since become a keystone technology, accelerating science and applications in a broad range of domains. Consequently, the notion of applying learning methods to a particular problem set has become an established and valuable modus operandi to advance a particular field. In this article we argue that such an approach does not straightforwardly extended to robotics -- or to embodied intelligence more generally: systems which engage in a purposeful exchange of energy and information with a physical environment. In particular, the purview of embodied intelligent agents extends significantly beyond the typical considerations of main-stream machine learning approaches, which typically (i) do not consider operation under conditions significantly different from those encountered during training; (ii) do not consider the often substantial, long-lasting and potentially safety-critical nature of interactions during learning and deployment; (iii) do not require ready adaptation to novel tasks while at the same time (iv) effectively and efficiently curating and extending their models of the world through targeted and deliberate actions. In reality, therefore, these limitations result in learning-based systems which suffer from many of the same operational shortcomings as more traditional, engineering-based approaches when deployed on a robot outside a well defined, and often narrow operating envelope. Contrary to viewing embodied intelligence as another application domain for machine learning, here we argue that it is in fact a key driver for the advancement of machine learning technology. In this article our goal is to highlight challenges and opportunities that are specific to embodied intelligence and to propose research directions which may significantly advance the state-of-the-art in robot learning.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 91 of its 191 references that are in the map. First posted on arXiv 28 Oct 2021.

Builds on

91 of 191 references.

References · 91

  1. 2017Attention Is All You Needcited by 60,361
  2. 2013Intriguing properties of neural networkscited by 5,155
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  6. 2013Auto-Encoding Variational Bayescited by 4,053
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  9. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  10. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  11. 2018World Modelscited by 1,513
  12. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
38 more
  1. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  2. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  3. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  4. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  5. 2018Learning Dexterous In-Hand Manipulationcited by 687
  6. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  7. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  8. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  9. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 392
  10. 2018GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoderscited by 383
  11. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  12. 2018Towards a Definition of Disentangled Representationscited by 296
  13. 2020Integrated Task and Motion Planningcited by 290
  14. 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 281
  15. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
  16. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  17. 2016Variational Intrinsic Controlcited by 267
  18. 2019Compositionality decomposed: how do neural networks generalise?cited by 265
  19. 2017Thinking Fast and Slow with Deep Learning and Tree Searchcited by 252
  20. 2019Learning Discrete Structures for Graph Neural Networkscited by 231
  21. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  22. 2019Recurrent Independent Mechanismscited by 219
  23. 2017Learning Explanatory Rules from Noisy Datacited by 211
  24. 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
  25. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  26. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  27. 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 176
  28. 2019SATNet: Bridging deep learning and logical reasoning using a differentiable satisfiability solvercited by 173
  29. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  30. 2016Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Datacited by 156
  31. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 155
  32. 2019Continual Unsupervised Representation Learningcited by 147
  33. 2019Neural Logic Machinescited by 145
  34. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  35. 2020Physically Realizable Adversarial Examples for LiDAR Object Detectioncited by 137
  36. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  37. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  38. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 8; 2024: 9; 2025: 9; 2026: 5.

35 papers.

Sort:

By area:

  1. 2023Levels of AGI for Operationalizing Progress on the Path to AGIcited by 96
  2. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  3. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
  4. 2023Advances in Embodied Navigation Using Large Language Models: A Surveycited by 14
  5. 2024From Efficient Multimodal Models to World Models: A Surveycited by 13
  6. 2022Tree Detection and Diameter Estimation Based on Deep Learningcited by 12
  7. 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
  8. 2025CausalStep: A Benchmark for Explicit Stepwise Causal Reasoning in Videoscited by 9
  9. 2023WALL-E: Embodied Robotic WAiter Load Lifting with Large Language Modelcited by 6
  10. 2024Language-Driven Visual Consensus for Zero-Shot Semantic Segmentationcited by 6
  11. 2024Polaris: Open-ended Interactive Robotic Manipulation via Syn2Real Visual Grounding and Large Language Modelscited by 6
  12. 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distancecited by 6
23 more
  1. 2026Ratio-Variance Regularized Policy Optimization for Efficient LLM Fine-tuningcited by 6
  2. 2022The Risks of Machine Learning Systemscited by 5
  3. 2023Invariance is Key to Generalization: Examining the Role of Representation in Sim-to-Real Transfer for Visual Navigationcited by 4
  4. 2025TongSIM: A General Platform for Simulating Intelligent Machinescited by 4
  5. 2021Social Neuro AI: Social Interaction as the "dark matter" of AIcited by 3
  6. 2022Provably Efficient Lifelong Reinforcement Learning with Linear Function Approximationcited by 2
  7. 2023Learning Type-Generalized Actions for Symbolic Planningcited by 2
  8. 2023Compositional Learning of Visually-Grounded Concepts Using Reinforcementcited by 2
  9. 2025Ark: An Open-source Python-based Framework for Robot Learningcited by 2
  10. 2025Multimodal Data Storage and Retrieval for Embodied AI: A Surveycited by 2
  11. 2025DINOv3-Diffusion Policy: Self-Supervised Large Visual Model for Visuomotor Diffusion Policy Learningcited by 2
  12. 2023Enactive Artificial Intelligence: Subverting Gender Norms in Robot-Human Interactioncited by 1
  13. 2023Scaling Distributed Multi-task Reinforcement Learning with Experience Sharingcited by 1
  14. 2024Sparse and Transferable Universal Singular Vectors Attackcited by 1
  15. 2024Bidirectional Progressive Neural Networks with Episodic Return Progress for Emergent Task Sequencing and Robotic Skill Transfercited by 1
  16. 2025Embodied Intelligence: The Key to Unblocking Generalized Artificial Intelligencecited by 1
  17. 2026FlowHijack: A Dynamics-Aware Backdoor Attack on Flow-Matching Vision-Language-Action Modelscited by 1
  18. 2026VEOcc: Voxel-Centric Online Semantic Occupancy Prediction For Embodied Scene Understandingcited by 1
  19. 2024OnDev-LCT: On-Device Lightweight Convolutional Transformers towards federated learningcited by 0
  20. 2024Generative AI Systems: A Systems-based Perspective on Generative AIcited by 0
  21. 2025Situating AI Agents in their World: Aspective Agentic AI for Dynamic Partially Observable Information Systemscited by 0
  22. 2026MetaSpace: Metamorphic Testing for Spatial Cognition in Embodied Agentscited by 0
  23. 2026Is Success All You Need? Investigating the Impact of Input Perturbations on VLA Behaviour in Tabletop Manipulation Taskscited by 0