# Distributed On-Sensor Compute System for AR/VR Devices: A Semi-Analytical Simulation Framework for Power Estimation

arXiv: https://arxiv.org/abs/2203.07474 · HTML: https://mlsanity.com/paper/2203.07474/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 18 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 2 of 2)

- [DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs](https://mlsanity.com/paper/2008.07127/) (2020)
- [XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference](https://mlsanity.com/paper/1807.03010/) (2018)

## Built on by (most cited first, 8 of 8)

- [PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors](https://mlsanity.com/paper/2304.05440/) (2023)
- [On-device Online Learning and Semantic Management of TinyML Systems](https://mlsanity.com/paper/2405.07601/) (2024)
- [MEMA Runtime Framework: Minimizing External Memory Accesses for TinyML on Microcontrollers](https://mlsanity.com/paper/2304.05544/) (2023)
- [SeLoC-ML: Semantic Low-Code Engineering for Machine Learning Applications in Industrial IoT](https://mlsanity.com/paper/2207.08818/) (2022)
- [Enabling ISP-less Low-Power Computer Vision](https://mlsanity.com/paper/2210.05451/) (2022)
- [Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices](https://mlsanity.com/paper/2410.08326/) (2024)
- [SteROI-D: System Design and Mapping for Stereo Depth Inference on Regions of Interest](https://mlsanity.com/paper/2502.09528/) (2025)
- [Hardware-Aware Learned Representation Compression for Distributed In-Sensor Vision](https://mlsanity.com/paper/2609.13947/) (2026)

## Abstract (arXiv)

Augmented Reality/Virtual Reality (AR/VR) glasses are widely foreseen as the next generation computing platform. AR/VR glasses are a complex "system of systems" which must satisfy stringent form factor, computing-, power- and thermal- requirements. In this paper, we will show that a novel distributed on-sensor compute architecture, coupled with new semiconductor technologies (such as dense 3D-IC interconnects and Spin-Transfer Torque Magneto Random Access Memory, STT-MRAM) and, most importantly, a full hardware-software co-optimization are the solutions to achieve attractive and socially acceptable AR/VR glasses. To this end, we developed a semi-analytical simulation framework to estimate the power consumption of novel AR/VR distributed on-sensor computing architectures. The model allows the optimization of the main technological features of the system modules, as well as the computer-vision algorithm partition strategy across the distributed compute architecture. We show that, in the case of the compute-intensive machine learning based Hand Tracking algorithm, the distributed on-sensor compute architecture can reduce the system power consumption compared to a centralized system, with the additional benefits in terms of latency and privacy.