# MSight: An Edge-Cloud Infrastructure-based Perception System for Connected Automated Vehicles

arXiv: https://arxiv.org/abs/2310.05290 · HTML: https://mlsanity.com/paper/2310.05290/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 76 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Video Object Segmentation and Tracking: A Survey](https://mlsanity.com/paper/1904.09172/) (2019)
- [GANet: Goal Area Network for Motion Forecasting](https://mlsanity.com/paper/2209.09723/) (2022)
- [A Vision-based System for Traffic Anomaly Detection using Deep Learning and Decision Trees](https://mlsanity.com/paper/2104.06856/) (2021)
- [PillarGrid: Deep Learning-based Cooperative Perception for 3D Object Detection from Onboard-Roadside LiDAR](https://mlsanity.com/paper/2203.06319/) (2022)
- [Waymo Public Road Safety Performance Data](https://mlsanity.com/paper/2011.00038/) (2020)
- [Conditional DETR V2: Efficient Detection Transformer with Box Queries](https://mlsanity.com/paper/2207.08914/) (2022)
- [Real-time Full-stack Traffic Scene Perception for Autonomous Driving with Roadside Cameras](https://mlsanity.com/paper/2206.09770/) (2022)
- [A Vision of C-V2X: Technologies, Field Testing and Challenges with Chinese Development](https://mlsanity.com/paper/2002.08736/) (2020)
- [Demonstrations of Cooperative Perception: Safety and Robustness in Connected and Automated Vehicle Operations](https://mlsanity.com/paper/2011.08581/) (2020)
- [Robust Roadside Perception: an Automated Data Synthesis Pipeline Minimizing Human Annotation](https://mlsanity.com/paper/2306.17302/) (2023)
- [Gabor Wavelets in Image Processing](https://mlsanity.com/paper/1602.03308/) (2016)

## Built on by (most cited first, 1 of 1)

- [Mcity Data Engine: Iterative Model Improvement Through Open-Vocabulary Data Selection](https://mlsanity.com/paper/2504.21614/) (2025)

## Abstract (arXiv)

As vehicular communication and networking technologies continue to advance, infrastructure-based roadside perception emerges as a pivotal tool for connected automated vehicle (CAV) applications. Due to their elevated positioning, roadside sensors, including cameras and lidars, often enjoy unobstructed views with diminished object occlusion. This provides them a distinct advantage over onboard perception, enabling more robust and accurate detection of road objects. This paper presents MSight, a cutting-edge roadside perception system specifically designed for CAVs. MSight offers real-time vehicle detection, localization, tracking, and short-term trajectory prediction. Evaluations underscore the system's capability to uphold lane-level accuracy with minimal latency, revealing a range of potential applications to enhance CAV safety and efficiency. Presently, MSight operates 24/7 at a two-lane roundabout in the City of Ann Arbor, Michigan.