# LaPlaSS: Latent Space Planning for Stochastic Systems

arXiv: https://arxiv.org/abs/2404.07063 · HTML: https://mlsanity.com/paper/2404.07063/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 10 Apr 2024.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 15 of 15)

- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Integrated Task and Motion Planning](https://mlsanity.com/paper/2010.01083/) (2020)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data](https://mlsanity.com/paper/1605.06432/) (2016)
- [A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning](https://mlsanity.com/paper/1710.05741/) (2017)
- [Learning Compositional Koopman Operators for Model-Based Control](https://mlsanity.com/paper/1910.08264/) (2019)
- [Temporal Predictive Coding For Model-Based Planning In Latent Space](https://mlsanity.com/paper/2106.07156/) (2021)
- [Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models](https://mlsanity.com/paper/1711.05772/) (2017)
- [Deep Variational Koopman Models: Inferring Koopman Observations for Uncertainty-Aware Dynamics Modeling and Control](https://mlsanity.com/paper/1902.09742/) (2019)
- [Fast Risk Assessment for Autonomous Vehicles Using Learned Models of Agent Futures](https://mlsanity.com/paper/2005.13458/) (2020)
- [Non-Gaussian Risk Bounded Trajectory Optimization for Stochastic Nonlinear Systems in Uncertain Environments](https://mlsanity.com/paper/2203.03038/) (2022)
- [Gradient-Based Mixed Planning with Symbolic and Numeric Action Parameters](https://mlsanity.com/paper/2110.10007/) (2021)
- [Optimal Mixed Discrete-Continuous Planning for Linear Hybrid Systems](https://mlsanity.com/paper/2102.08261/) (2021)
- [Evolutionary Planning in Latent Space](https://mlsanity.com/paper/2011.11293/) (2020)

## Built on by (most cited first, 1 of 1)

- [MoMo: Conditioned Contrastive Representation Learning for Preference-Modulated Planning](https://mlsanity.com/paper/2605.08512/) (2026)

## Abstract (arXiv)

Autonomous mobile agents often operate in hazardous environments, necessitating an awareness of safety. These agents can have non-linear, stochastic dynamics that must be considered during planning to guarantee bounded risk. Most state of the art methods require closed-form dynamics to verify plan correctness and safety however modern robotic systems often have dynamics that are learned from data. Thus, there is a need to perform efficient trajectory planning with guarantees on risk for agents without known dynamics models. We propose a "generate-and-test" approach to risk-bounded planning in which a planner generates a candidate trajectory using an approximate linear dynamics model and a validator assesses the risk of the trajectory, computing additional safety constraints for the planner if the candidate does not satisfy the desired risk bound. To acquire the approximate model, we use a variational autoencoder to learn a latent linear dynamics model and encode the planning problem into the latent space to generate the candidate trajectory. The VAE also serves to sample trajectories around the candidate to use in the validator. We demonstrate that our algorithm, LaPlaSS, is able to generate trajectory plans with bounded risk for a real-world agent with learned dynamics and is an order of magnitude more efficient than the state of the art.