mlsanityData to 14 Oct 2026

MIND-Stack: Modular, Interpretable, End-to-End Differentiability for Autonomous Navigation

arXiv 2505.21734 ↗First posted 27 May 2025cs.RO cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Developing robust, efficient navigation algorithms is challenging. Rule-based methods offer interpretability and modularity but struggle with learning from large datasets, while end-to-end neural networks excel in learning but lack transparency and modularity. In this paper, we present MIND-Stack, a modular software stack consisting of a localization network and a Stanley Controller with intermediate human interpretable state representations and end-to-end differentiability. Our approach enables the upstream localization module to reduce the downstream control error, extending its role beyond state estimation. Unlike existing research on differentiable algorithms that either lack modules of the autonomous stack to span from sensor input to actuator output or real-world implementation, MIND-Stack offers both capabilities. We conduct experiments that demonstrate the ability of the localization module to reduce the downstream control loss through its end-to-end differentiability while offering better performance than state-of-the-art algorithms. We showcase sim-to-real capabilities by deploying the algorithm on a real-world embedded autonomous platform with limited computation power and demonstrate simultaneous training of both the localization and controller towards one goal. While MIND-Stack shows good results, we discuss the incorporation of additional modules from the autonomous navigation pipeline in the future, promising even greater stability and performance in the next iterations of the framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

33 of 44 references.

References · 33

  1. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  2. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  3. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  4. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 403
  5. 2018Learning to Drive in a Daycited by 273
  6. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 240
  7. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  8. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 149
  9. 2022Learning from All Vehiclescited by 135
  10. 2019Multimodal End-to-End Autonomous Drivingcited by 110
  11. 2017Interpretable Learning for Self-Driving Cars by Visualizing Causal Attentioncited by 107
  12. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 103
21 more
  1. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 103
  2. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  3. 2021Explanations in Autonomous Driving: A Surveycited by 81
  4. 2022Deep Model Reassemblycited by 78
  5. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directionscited by 73
  6. 2022Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racingcited by 64
  7. 2019Pontryagin Differentiable Programming: An End-to-End Learning and Control Frameworkcited by 52
  8. 2022Real-time Neural-MPC: Deep Learning Model Predictive Control for Quadrotors and Agile Robotic Platformscited by 41
  9. 2022DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehiclescited by 37
  10. 2022PyPose: A Library for Robot Learning with Physics-based Optimizationcited by 29
  11. 2019Differentiable Algorithm Networks for Composable Robot Learningcited by 24
  12. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 17
  13. 2021Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learningcited by 11
  14. 2023High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learningcited by 10
  15. 2017CDDT: Fast Approximate 2D Ray Casting for Accelerated Localizationcited by 9
  16. 2024Unifying F1TENTH Autonomous Racing: Survey, Methods and Benchmarkscited by 8
  17. 2022Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Drivingcited by 7
  18. 2023PyPose v0.6: The Imperative Programming Interface for Roboticscited by 7
  19. 2019Deep Learning Based Motion Planning For Autonomous Vehicle Using Spatiotemporal LSTM Networkcited by 4
  20. 2023Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Drivingcited by 4
  21. 2022Local_INN: Implicit Map Representation and Localization with Invertible Neural Networkscited by 2

Built on by

Not cited yet.