# Heterogeneous computing platform for real-time robotics

arXiv: https://arxiv.org/abs/2601.09755 · HTML: https://mlsanity.com/paper/2601.09755/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 58 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Neurosymbolic AI: The 3rd Wave](https://mlsanity.com/paper/2012.05876/) (2020)
- [The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence](https://mlsanity.com/paper/2002.06177/) (2020)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [Efficient Neuromorphic Signal Processing with Loihi 2](https://mlsanity.com/paper/2111.03746/) (2021)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges](https://mlsanity.com/paper/2112.15424/) (2021)
- [A comparison of Vector Symbolic Architectures](https://mlsanity.com/paper/2001.11797/) (2020)
- [Classification using Hyperdimensional Computing: A Review](https://mlsanity.com/paper/2004.11204/) (2020)
- [Neuromorphic Intermediate Representation: A Unified Instruction Set for Interoperable Brain-Inspired Computing](https://mlsanity.com/paper/2311.14641/) (2023)
- [Neuromorphic Visual Scene Understanding with Resonator Networks](https://mlsanity.com/paper/2208.12880/) (2022)
- [How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites](https://mlsanity.com/paper/1601.00720/) (2016)
- [Embodied AI-Driven Operation of Smart Cities: A Concise Review](https://mlsanity.com/paper/2108.09823/) (2021)
- [DAC-h3: A Proactive Robot Cognitive Architecture to Acquire and Express Knowledge About the World and the Self](https://mlsanity.com/paper/1706.03661/) (2017)
- [Real-Time Cortical Simulation on Neuromorphic Hardware](https://mlsanity.com/paper/1909.08665/) (2019)
- [Distributed Impedance Control of Latency-Prone Robotic Systems with Series Elastic Actuation](https://mlsanity.com/paper/1811.11573/) (2018)

## Abstract (arXiv)

After Industry 4.0 has embraced tight integration between machinery (OT), software (IT), and the Internet, creating a web of sensors, data, and algorithms in service of efficient and reliable production, a new concept of Society 5.0 is emerging, in which infrastructure of a city will be instrumented to increase reliability, efficiency, and safety. Robotics will play a pivotal role in enabling this vision that is pioneered by the NEOM initiative - a smart city, co-inhabited by humans and robots. In this paper we explore the computing platform that will be required to enable this vision. We show how we can combine neuromorphic computing hardware, exemplified by the Loihi2 processor used in conjunction with event-based cameras, for sensing and real-time perception and interaction with a local AI compute cluster (GPUs) for high-level language processing, cognition, and task planning. We demonstrate the use of this hybrid computing architecture in an interactive task, in which a humanoid robot plays a musical instrument with a human. Central to our design is the efficient and seamless integration of disparate components, ensuring that the synergy between software and hardware maximizes overall performance and responsiveness. Our proposed system architecture underscores the potential of heterogeneous computing architectures in advancing robotic autonomy and interactive intelligence, pointing toward a future where such integrated systems become the norm in complex, real-time applications.