# Deployment of Deep Neural Networks for Object Detection on Edge AI Devices with Runtime Optimization

arXiv: https://arxiv.org/abs/2108.08166 · HTML: https://mlsanity.com/paper/2108.08166/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 18 Aug 2021.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation](https://mlsanity.com/paper/2004.09602/) (2020)
- [SemanticVoxels: Sequential Fusion for 3D Pedestrian Detection using LiDAR Point Cloud and Semantic Segmentation](https://mlsanity.com/paper/2009.12276/) (2020)

## Built on by (most cited first, 8 of 8)

- [LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection](https://mlsanity.com/paper/2401.15865/) (2024)
- [Information Entropy Guided Height-aware Histogram for Quantization-friendly Pillar Feature Encoder](https://mlsanity.com/paper/2405.18734/) (2024)
- [Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems](https://mlsanity.com/paper/2207.14694/) (2022)
- [Data-Model-Circuit Tri-Design for Ultra-Light Video Intelligence on Edge Devices](https://mlsanity.com/paper/2210.08578/) (2022)
- [AI in Space for Scientific Missions: Strategies for Minimizing Neural-Network Model Upload](https://mlsanity.com/paper/2406.14297/) (2024)
- [3D Harmonic Loss: Towards Task-consistent and Time-friendly 3D Object Detection on Edge for V2X Orchestration](https://mlsanity.com/paper/2211.03407/) (2022)
- [A Framework for Benchmarking Real-Time Embedded Object Detection](https://mlsanity.com/paper/2304.11580/) (2023)
- [Real-time Multi-modal Object Detection and Tracking on Edge for Regulatory Compliance Monitoring](https://mlsanity.com/paper/2310.03333/) (2023)

## Abstract (arXiv)

Deep neural networks have proven increasingly important for automotive scene understanding with new algorithms offering constant improvements of the detection performance. However, there is little emphasis on experiences and needs for deployment in embedded environments. We therefore perform a case study of the deployment of two representative object detection networks on an edge AI platform. In particular, we consider RetinaNet for image-based 2D object detection and PointPillars for LiDAR-based 3D object detection. We describe the modifications necessary to convert the algorithms from a PyTorch training environment to the deployment environment taking into account the available tools. We evaluate the runtime of the deployed DNN using two different libraries, TensorRT and TorchScript. In our experiments, we observe slight advantages of TensorRT for convolutional layers and TorchScript for fully connected layers. We also study the trade-off between runtime and performance, when selecting an optimized setup for deployment, and observe that quantization significantly reduces the runtime while having only little impact on the detection performance.