mlsanityData to 7 Oct 2026

Pylot: A Modular Platform for Exploring Latency-Accuracy Tradeoffs in Autonomous Vehicles

arXiv 2104.07830 ↗First posted 16 Apr 2021cs.ROcited by 33

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

We present Pylot, a platform for autonomous vehicle (AV) research and development, built with the goal to allow researchers to study the effects of the latency and accuracy of their models and algorithms on the end-to-end driving behavior of an AV. This is achieved through a modular structure enabled by our high-performance dataflow system that represents AV software pipeline components (object detectors, motion planners, etc.) as a dataflow graph of operators which communicate on data streams using timestamped messages. Pylot readily interfaces with popular AV simulators like CARLA, and is easily deployable to real-world vehicles with minimal code changes. To reduce the burden of developing an entire pipeline for evaluating a single component, Pylot provides several state-of-the-art reference implementations for the various components of an AV pipeline. Using these reference implementations, a Pylot-based AV pipeline is able to drive a real vehicle, and attains a high score on the CARLA Autonomous Driving Challenge. We also present several case studies enabled by Pylot, including evidence of a need for context-dependent components, and per-component time allocation. Pylot is open source, with the code available at https://github.com/erdos-project/pylot.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 26 of its 63 references that are in the map. First posted on arXiv 16 Apr 2021.

Builds on

26 of 63 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021End-to-end Interpretable Neural Motion Planner17 Jan

Other references · 25

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2016OpenAI Gymcited by 1,684
  8. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  10. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,018
  11. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  12. 2016Simple Online and Realtime Trackingcited by 862
13 more
  1. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  2. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  3. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  4. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
  5. 2016Speed/accuracy trade-offs for modern convolutional object detectorscited by 198
  6. 2018Distractor-aware Siamese Networks for Visual Object Trackingcited by 187
  7. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  8. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  9. 2019Multiple Futures Predictioncited by 178
  10. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  11. 2020LGSVL Simulator: A High Fidelity Simulator for Autonomous Drivingcited by 119
  12. 2020Learning to Evaluate Perception Models Using Planner-Centric Metricscited by 49
  13. 2018Control of Connected and Automated Vehicles: State of the Art and Future Challengescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 9; 2024: 7; 2025: 6; 2026: 5.

33 papers.

Sort:

By area:

  1. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  2. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  3. 2023Does Physical Adversarial Example Really Matter to Autonomous Driving? Towards System-Level Effect of Adversarial Object Evasion Attackcited by 37
  4. 2022MMFN: Multi-Modal-Fusion-Net for End-to-End Drivingcited by 17
  5. 2024Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMscited by 13
  6. 2023SuperServe: Fine-Grained Inference Serving for Unpredictable Workloadscited by 12
  7. 2022Context-Aware Streaming Perception in Dynamic Environmentscited by 8
  8. 2021ViSTA: a Framework for Virtual Scenario-based Testing of Autonomous Vehiclescited by 6
  9. 2023RED: A Systematic Real-Time Scheduling Approach for Robotic Environmental Dynamicscited by 4
  10. 2023Evaluating the Impact of Flaky Simulators on Testing Autonomous Driving Systemscited by 4
  11. 2024Decictor: Towards Evaluating the Robustness of Decision-Making in Autonomous Driving Systemscited by 4
  12. 2025LoGoPlanner: Localization Grounded Navigation Policy with Metric-aware Visual Geometrycited by 4
21 more
  1. 2024MaskFuser: Masked Fusion of Joint Multi-Modal Tokenization for End-to-End Autonomous Drivingcited by 3
  2. 2024VALO: A Versatile Anytime Framework for LiDAR-based Object Detection Deep Neural Networkscited by 3
  3. 2023Identifying the Hazard Boundary of ML-enabled Autonomous Systems Using Cooperative Co-Evolutionary Searchcited by 2
  4. 2023Interpretable End-to-End Driving Model for Implicit Scene Understandingcited by 2
  5. 2025A Novel Multi-layer Task-centric and Data Quality Framework for Autonomous Drivingcited by 2
  6. 2022Watch Out for the Safety-Threatening Actors: Proactively Mitigating Safety Hazardscited by 1
  7. 2023Reasoning-Based Software Testingcited by 1
  8. 2024DεpS: Delayed ε-Shrinking for Faster Once-For-All Trainingcited by 1
  9. 2024Managing Bandwidth: The Key to Cloud-Assisted Autonomous Drivingcited by 1
  10. 2025Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarioscited by 1
  11. 2025ADReFT: Adaptive Decision Repair for Safe Autonomous Driving via Reinforcement Fine-Tuningcited by 1
  12. 2025A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systemscited by 1
  13. 2023SEO: Safety-Aware Energy Optimization Framework for Multi-Sensor Neural Controllers at the Edgecited by 0
  14. 2023A Framework for Benchmarking Real-Time Embedded Object Detectioncited by 0
  15. 2024Tuning the feedback controller gains is a simple way to improve autonomous driving performancecited by 0
  16. 2025STCLocker: Deadlock Avoidance Testing for Autonomous Driving Systemscited by 0
  17. 2026Multi-Resolution End-to-End Deep Neural Network for Optimizing Latency-Accuracy Tradeoff in Autonomous Drivingcited by 0
  18. 2026CADET: A Modular Platform for Evaluating Distributed Cooperative Autonomy in Connected Autonomous Vehiclescited by 0
  19. 2026On Exploring Input Resolution Scaling For Anytime LiDAR Object Detectioncited by 0
  20. 2026LLM-Assisted Dynamic Threat Analysis for Attacker-Reachable Software Weaknesses in Autonomous Vehiclescited by 0
  21. 2026Correcting Learning-based Perception for Safetycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Pseudo-Simulation for Autonomous Driving
  2. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions
  3. 2021Autonomy 2.0: Why is self-driving always 5 years away?
  4. 2025Data Scaling Laws for End-to-End Autonomous Driving
  5. 2025PlanT 2.0: Exposing Biases and Structural Flaws in Closed-Loop Driving
  6. 2026Drive the Thoughts: Runtime Monitoring of VLA Reasoning-Trajectory Consistency
  7. 2025X-Driver: Explainable Autonomous Driving with Vision-Language Models
  8. 2026DriveVer: Lightweight Trajectory Evaluator as Test-Time Verifier for Autonomous Driving