# CARMA: Context-Aware Runtime Reconfiguration for Energy-Efficient Sensor Fusion

arXiv: https://arxiv.org/abs/2306.15748 · HTML: https://mlsanity.com/paper/2306.15748/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 10 of 10)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [RADIATE: A Radar Dataset for Automotive Perception in Bad Weather](https://mlsanity.com/paper/2010.09076/) (2020)
- [FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge](https://mlsanity.com/paper/1904.04421/) (2019)
- [HydraFusion: Context-Aware Selective Sensor Fusion for Robust and Efficient Autonomous Vehicle Perception](https://mlsanity.com/paper/2201.06644/) (2022)
- [EcoFusion: Energy-Aware Adaptive Sensor Fusion for Efficient Autonomous Vehicle Perception](https://mlsanity.com/paper/2202.11330/) (2022)
- [SAGE: A Split-Architecture Methodology for Efficient End-to-End Autonomous Vehicle Control](https://mlsanity.com/paper/2107.10895/) (2021)
- [Opportunistic Intermittent Control with Safety Guarantees for Autonomous Systems](https://mlsanity.com/paper/2005.03726/) (2020)
- [FEEL: Fast, Energy-Efficient Localization for Autonomous Indoor Vehicles](https://mlsanity.com/paper/2102.00702/) (2021)

## Abstract (arXiv)

Autonomous systems (AS) are systems that can adapt and change their behavior in response to unanticipated events and include systems such as aerial drones, autonomous vehicles, and ground/aquatic robots. AS require a wide array of sensors, deep-learning models, and powerful hardware platforms to perceive and safely operate in real-time. However, in many contexts, some sensing modalities negatively impact perception while increasing the system's overall energy consumption. Since AS are often energy-constrained edge devices, energy-efficient sensor fusion methods have been proposed. However, existing methods either fail to adapt to changing scenario conditions or to optimize energy efficiency system-wide. We propose CARMA: a context-aware sensor fusion approach that uses context to dynamically reconfigure the computation flow on a Field-Programmable Gate Array (FPGA) at runtime. By clock-gating unused sensors and model sub-components, CARMA significantly reduces the energy used by a multi-sensory object detector without compromising performance. We use a Deep-learning Processor Unit (DPU) based reconfiguration approach to minimize the latency of model reconfiguration. We evaluate multiple context-identification strategies, propose a novel system-wide energy-performance joint optimization, and evaluate scenario-specific perception performance. Across challenging real-world sensing contexts, CARMA outperforms state-of-the-art methods with up to 1.3x speedup and 73% lower energy consumption.