# CODEI: Resource-Efficient Task-Driven Co-Design of Perception and Decision Making for Mobile Robots Applied to Autonomous Vehicles

arXiv: https://arxiv.org/abs/2503.10296 · HTML: https://mlsanity.com/paper/2503.10296/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 19 of its 83 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 19 of 19)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [Dirichlet-based Gaussian Processes for Large-scale Calibrated Classification](https://mlsanity.com/paper/1805.10915/) (2018)
- [Optimization-Based Autonomous Racing of 1:43 Scale RC Cars](https://mlsanity.com/paper/1711.07300/) (2017)
- [Practical Design Space Exploration](https://mlsanity.com/paper/1810.05236/) (2018)
- [A Mathematical Theory of Co-Design](https://mlsanity.com/paper/1512.08055/) (2015)
- [Introducing SLAMBench, a performance and accuracy benchmarking methodology for SLAM](https://mlsanity.com/paper/1410.2167/) (2014)
- [A Review of Testing Object-Based Environment Perception for Safe Automated Driving](https://mlsanity.com/paper/2102.08460/) (2021)
- [Co-Design of Embodied Intelligence: A Structured Approach](https://mlsanity.com/paper/2011.10756/) (2020)
- [Co-Design of Autonomous Systems: From Hardware Selection to Control Synthesis](https://mlsanity.com/paper/2011.10758/) (2020)
- [Deterministic Sampling-Based Motion Planning: Optimality, Complexity, and Performance](https://mlsanity.com/paper/1505.00023/) (2015)
- [LQG Control and Sensing Co-Design](https://mlsanity.com/paper/1802.08376/) (2018)
- [Toward a language-theoretic foundation for planning and filtering](https://mlsanity.com/paper/1807.08856/) (2018)
- [Co-Design to Enable User-Friendly Tools to Assess the Impact of Future Mobility Solutions](https://mlsanity.com/paper/2008.08975/) (2020)
- [Robot Design: Formalisms, Representations, and the Role of the Designer](https://mlsanity.com/paper/1806.05157/) (2018)
- [Abstractions for computing all robotic sensors that suffice to solve a planning problem](https://mlsanity.com/paper/2005.10994/) (2020)
- [Task-driven Modular Co-design of Vehicle Control Systems](https://mlsanity.com/paper/2203.16640/) (2022)

## Built on by (most cited first, 1 of 1)

- [Uncertainty-Aware End-to-End Co-Design of Neural Network Processors: From Training and Mapping to Fabrication](https://mlsanity.com/paper/2606.04850/) (2026)

## Abstract (arXiv)

This paper discusses the integration challenges and strategies for designing mobile robots, by focusing on the task-driven, optimal selection of hardware and software to balance safety, efficiency, and minimal usage of resources such as costs, energy, computational requirements, and weight. We emphasize the interplay between perception and motion planning in decision-making by introducing the concept of occupancy queries to quantify the perception requirements for sampling-based motion planners. Sensor and algorithm performance are evaluated using False Negative Rates (FPR) and False Positive Rates (FPR) across various factors such as geometric relationships, object properties, sensor resolution, and environmental conditions. By integrating perception requirements with perception performance, an Integer Linear Programming (ILP) approach is proposed for efficient sensor and algorithm selection and placement. This forms the basis for a co-design optimization that includes the robot body, motion planner, perception pipeline, and computing unit. We refer to this framework for solving the co-design problem of mobile robots as CODEI, short for Co-design of Embodied Intelligence. A case study on developing an Autonomous Vehicle (AV) for urban scenarios provides actionable information for designers, and shows that complex tasks escalate resource demands, with task performance affecting choices of the autonomy stack. The study demonstrates that resource prioritization influences sensor choice: cameras are preferred for cost-effective and lightweight designs, while lidar sensors are chosen for better energy and computational efficiency.