# TAPAS: Throughput-adaptive Perception for Autonomous Systems

arXiv: https://arxiv.org/abs/2607.17317 · HTML: https://mlsanity.com/paper/2607.17317/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 19 Jul 2026.

## Builds on (most cited first, 22 of 22)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Reward Design with Language Models](https://mlsanity.com/paper/2303.00001/) (2023)
- [DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks](https://mlsanity.com/paper/1709.08429/) (2017)
- [Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity](https://mlsanity.com/paper/2210.09579/) (2022)
- [Reward Reasoning Model](https://mlsanity.com/paper/2505.14674/) (2025)
- [OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workload](https://mlsanity.com/paper/2307.03290/) (2023)
- [Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach](https://mlsanity.com/paper/2305.06121/) (2023)
- [Semantically-driven Deep Reinforcement Learning for Inspection Path Planning](https://mlsanity.com/paper/2505.14443/) (2025)
- [Deep Reinforcement Learning based Online Scheduling Policy for Deep Neural Network Multi-Tenant Multi-Accelerator Systems](https://mlsanity.com/paper/2404.08950/) (2024)
- [RoboRun: A Robot Runtime to Exploit Spatial Heterogeneity](https://mlsanity.com/paper/2108.13354/) (2021)
- [Energy-Efficient Mobile Robot Control via Run-time Monitoring of Environmental Complexity and Computing Workload](https://mlsanity.com/paper/2109.04285/) (2021)
- [Zhuyi: Perception Processing Rate Estimation for Safety in Autonomous Vehicles](https://mlsanity.com/paper/2205.03347/) (2022)
- [Equipping Diffusion Models with Differentiable Spatial Entropy for Low-Light Image Enhancement](https://mlsanity.com/paper/2404.09735/) (2024)
- [Context-aware Multi-Model Object Detection for Diversely Heterogeneous Compute Systems](https://mlsanity.com/paper/2402.07415/) (2024)
- [The use of spatial information in entropy measures](https://mlsanity.com/paper/1703.06001/) (2017)
- [Adaptive Optimization of Autonomous Vehicle Computational Resources for Performance and Energy Improvement](https://mlsanity.com/paper/2102.07000/) (2021)
- [Maritime Vessel Tank Inspection using Aerial Robots: Experience from the field and dataset release](https://mlsanity.com/paper/2404.19045/) (2024)

## Abstract (arXiv)

Autonomous systems rely on a perception module to navigate through dynamic environments. In real-world scenarios, the perception module's throughput requirements vary at runtime due to changes in scene complexity. However, existing perception strategies assume a fixed FPS and static model-to-cluster mapping, resulting in either over/under provision of throughput requirements or unnecessary energy consumption across diverse scenes. Addressing this challenge requires tightly coupled textitscene complexity awareness to estimate an appropriate FPS target and textitdynamic model-to-cluster mapping to deliver the required throughput at minimum energy. We propose a throughput-adaptive perception strategy for mobile/edge platforms, enabling intelligent runtime resource allocation based on varying FPS targets. We use Reinforcement Learning (RL) with RRM (Reward Reasoning Model) and a GRU (Gated Recurrent Unit) agent to orchestrate perception tasks across heterogeneous mobile/edge platforms. We evaluate TAPAS on Jetson Orin NX across KITTI and unseen nuScenes. On the textitKITTI dataset's test sequences, TAPAS achieves 93-100% throughput met rate while saving energy by 76%. On the unseen textitnuScenes dataset, TAPAS maintains 97% throughput met rate with 64% lower energy compared to textitSOTA approaches, proving its robustness.