LIDAR-AD: A Decoder-Free Latent-Interaction Dreamer with Action-Residual Chains for Autonomous Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
Autonomous driving requires long-horizon closedloop decision making in dynamic traffic environments. Latent world models offer an effective framework for this problem by enabling imagination-based decision making in compact latent spaces. However, multi-source observations contain controlirrelevant redundancy, whereas reliable driving decisions rely on risk-relevant relations, future dynamics, and continuous action adjustments. This mismatch makes observation reconstruction and absolute action modeling suboptimal for learning decisionrelevant latent dynamics. We propose LIDAR-AD, a decoderfree Latent-Interaction Dreamer with Action-Residual Chains for autonomous driving. LIDAR-AD replaces observation reconstruction with redundancy-reduced latent alignment, encouraging compact representations of risk-relevant relations in multi-source driving inputs. It further models vehicle control as residual action updates and uses residual-action sequence contrastive learning to align multi-step residual-driven rollouts with future latent states. A deterministic analysis shows that the latent-tanh residual parameterization preserves interior action reachability while representing smooth long-horizon control as compact local updates. Together, these designs improve risk-aware state abstraction, continuous-control modeling, and long-horizon dynamics prediction. Extensive experiments across diverse simulated driving scenarios demonstrate that LIDAR-AD consistently outperforms world-model baselines, achieving the highest reward and the best success rate among learning-based methods. Evaluations on nuPlan-derived log-reconstructed scenarios further demonstrate the transferability of LIDAR-AD under real-world traffic layouts.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 34 of its 38 references that are in the map. First posted on arXiv 12 Jul 2026.
Builds on
34 of 38 references.
Closest ancestors · 6
Same direction, newest first
- 2024Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)26 Feb
- 2024GenAD: Generative End-to-End Autonomous Driving18 Feb
- 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving18 Sep
- 2022Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model8 Oct
- 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
Other references · 28
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
16 more
- 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
- 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
- 2022Temporal Difference Learning for Model Predictive Controlcited by 375
- 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
- 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
- 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 251
- 2018Residual Reinforcement Learning for Robot Controlcited by 242
- 2022Model-Based Imitation Learning for Urban Drivingcited by 149
- 2023ScenarioNet: Open-Source Platform for Large-Scale Traffic Scenario Simulation and Modelingcited by 60
- 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 50
- 2020Regularizing Action Policies for Smooth Control with Reinforcement Learningcited by 47
- 2025Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulationcited by 16
- 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
- 2026ResWM: Residual-Action World Model for Visual RLcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
- 2026DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving
- 2026Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving
- 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Driving
- 2021Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learning
- 2026CLEAR: Cognition and Latent Evaluation for Adaptive Routing in End-to-End Autonomous Driving
- 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
- 2026WALT: Learning World-Model-Aligned Latent Trajectories for Autonomous Driving