# Digital Twin-based Control Co-Design of Full Vehicle Active Suspensions via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2512.03891 · HTML: https://mlsanity.com/paper/2512.03891/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 84 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 20 of 20)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Embodied Intelligence via Learning and Evolution](https://mlsanity.com/paper/2102.02202/) (2021)
- [Jointly Learning to Construct and Control Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/1801.01432/) (2018)
- [Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning](https://mlsanity.com/paper/1911.06832/) (2019)
- [Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.04460/) (2020)
- [A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies](https://mlsanity.com/paper/2208.14197/) (2022)
- [A Comprehensive Review of Digital Twin -- Part 2: Roles of Uncertainty Quantification and Optimization, a Battery Digital Twin, and Perspectives](https://mlsanity.com/paper/2208.12904/) (2022)
- [A Latent Variable Approach for Non-Hierarchical Multi-Fidelity Adaptive Sampling](https://mlsanity.com/paper/2310.03298/) (2023)
- [Towards a Digital Twin Framework in Additive Manufacturing: Machine Learning and Bayesian Optimization for Time Series Process Optimization](https://mlsanity.com/paper/2402.17718/) (2024)
- [Real-Time Decision-Making for Digital Twin in Additive Manufacturing with Model Predictive Control using Time-Series Deep Neural Networks](https://mlsanity.com/paper/2501.07601/) (2025)
- [4D topology optimization: Integrated optimization of the structure and self-actuation of soft bodies for dynamic motions](https://mlsanity.com/paper/2302.00905/) (2023)
- [Uncertainty-Aware Digital Twins: Robust Model Predictive Control using Time-Series Deep Quantile Learning](https://mlsanity.com/paper/2501.10337/) (2025)
- [Robust Control Co-Design with Receding-Horizon MPC](https://mlsanity.com/paper/2104.02025/) (2021)
- [C^2:Co-design of Robots via Concurrent Networks Coupling Online and Offline Reinforcement Learning](https://mlsanity.com/paper/2209.06579/) (2022)
- [Computational co-design of structure and feedback controller for locomoting soft robots](https://mlsanity.com/paper/2407.09270/) (2024)

## Abstract (arXiv)

Active suspension systems are critical for enhancing vehicle comfort, safety, and stability, yet their performance is often limited by fixed hardware designs and control strategies that cannot adapt to uncertain and dynamic operating conditions. Recent advances in digital twins (DTs) and deep reinforcement learning (DRL) offer new opportunities for real-time, data-driven optimization across a vehicle's lifecycle. However, integrating these technologies into a unified framework remains an open challenge. This work presents a DT-based control co-design (CCD) framework for full-vehicle active suspensions using multi-generation design concepts. By integrating automatic differentiation into DRL, we jointly optimize physical suspension components and control policies under varying driver behaviors and environmental uncertainties. DRL also addresses the challenge of partial observability, where only limited states can be sensed and fed back to the controller, by learning optimal control actions directly from available sensor information. The framework incorporates model updating with quantile learning to capture data uncertainty, enabling real-time decision-making and adaptive learning from digital-physical interactions. The approach demonstrates personalized optimization of suspension systems under two distinct driving settings (mild and aggressive). Results show that the optimized systems achieve smoother trajectories and reduce control efforts by approximately 43% and 52% for mild and aggressive, respectively, while maintaining ride comfort and stability. Contributions include: developing a DT-enabled CCD framework integrating DRL and uncertainty-aware model updating for full-vehicle active suspensions, introducing a multi-generation design strategy for self-improving systems, and demonstrating personalized optimization of active suspension systems for distinct driver types.