# RoboBrain: Large-Scale Knowledge Engine for Robots

arXiv: https://arxiv.org/abs/1412.0691 · HTML: https://mlsanity.com/paper/1412.0691/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2014.

## Built on by (most cited first, 13 of 13)

- [Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention](https://mlsanity.com/paper/1812.04155/) (2018)
- [Continual Learning of Knowledge Graph Embeddings](https://mlsanity.com/paper/2101.05850/) (2021)
- [Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI](https://mlsanity.com/paper/2311.03783/) (2023)
- [Aligning Robot and Human Representations](https://mlsanity.com/paper/2302.01928/) (2023)
- [Towards Robust One-shot Task Execution using Knowledge Graph Embeddings](https://mlsanity.com/paper/2105.04484/) (2021)
- [Building Affordance Relations for Robotic Agents - A Review](https://mlsanity.com/paper/2105.06706/) (2021)
- [Safety Control of Service Robots with LLMs and Embodied Knowledge Graphs](https://mlsanity.com/paper/2405.17846/) (2024)
- [Generalizable task representation learning from human demonstration videos: a geometric approach](https://mlsanity.com/paper/2202.13604/) (2022)
- [Graph learning in robotics: a survey](https://mlsanity.com/paper/2310.04294/) (2023)
- [Metarobotics for Industry and Society: Vision, Technologies, and Opportunities](https://mlsanity.com/paper/2404.00797/) (2024)
- [Many Episode Learning in a Modular Embodied Agent via End-to-End Interaction](https://mlsanity.com/paper/2204.08687/) (2022)
- [From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence](https://mlsanity.com/paper/2607.11689/) (2026)
- [tinyDSM: A Framework for Skill Modeling and Development for Resource-Constrained Millirobots](https://mlsanity.com/paper/2608.17596/) (2026)

## Abstract (arXiv)

In this paper we introduce a knowledge engine, which learns and shares knowledge representations, for robots to carry out a variety of tasks. Building such an engine brings with it the challenge of dealing with multiple data modalities including symbols, natural language, haptic senses, robot trajectories, visual features and many others. The textitknowledge stored in the engine comes from multiple sources including physical interactions that robots have while performing tasks (perception, planning and control), knowledge bases from the Internet and learned representations from several robotics research groups. We discuss various technical aspects and associated challenges such as modeling the correctness of knowledge, inferring latent information and formulating different robotic tasks as queries to the knowledge engine. We describe the system architecture and how it supports different mechanisms for users and robots to interact with the engine. Finally, we demonstrate its use in three important research areas: grounding natural language, perception, and planning, which are the key building blocks for many robotic tasks. This knowledge engine is a collaborative effort and we call it RoboBrain.