# Adversarial observations in probabilistic State-Space Models for robust Reinforcement Learning

arXiv: https://arxiv.org/abs/2606.20880 · HTML: https://mlsanity.com/paper/2606.20880/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [Structured State Space Models for In-Context Reinforcement Learning](https://mlsanity.com/paper/2303.03982/) (2023)
- [Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning](https://mlsanity.com/paper/2205.01992/) (2022)
- [Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks](https://mlsanity.com/paper/1701.04143/) (2017)
- [Outlier-robust Kalman Filtering through Generalised Bayes](https://mlsanity.com/paper/2405.05646/) (2024)
- [Wasserstein Distributionally Robust Kalman Filtering](https://mlsanity.com/paper/1809.08830/) (2018)
- [Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mamba](https://mlsanity.com/paper/2503.18970/) (2025)
- [Adversarial Machine Learning: Bayesian Perspectives](https://mlsanity.com/paper/2003.03546/) (2020)
- [Adversarial Inception Backdoor Attacks against Reinforcement Learning](https://mlsanity.com/paper/2410.13995/) (2024)
- [Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2409.16824/) (2024)
- [A unifying framework for generalised Bayesian online learning in non-stationary environments](https://mlsanity.com/paper/2411.10153/) (2024)
- [Secure State Estimation For Cyber Physical Systems Under Sensor Attacks: A Satisfiability Modulo Theory Approach](https://mlsanity.com/paper/1412.4324/) (2014)
- [Variational Bayesian Adaptation of Noise Covariances in Non-Linear Kalman Filtering](https://mlsanity.com/paper/1302.0681/) (2013)
- [Robust Bayesian Filtering and Smoothing Using Student's t Distribution](https://mlsanity.com/paper/1703.02428/) (2017)
- [Distributionally Robust Kalman Filtering over Finite and Infinite Horizon](https://mlsanity.com/paper/2407.18837/) (2024)

## Abstract (arXiv)

Decision-making under partial or adversarial observability requires accurate inference of the environment's latent state and its associated uncertainty. This work analyses adversarial attacks on linear state-space models, where the attacker alters observations subject to likelihood constraints that ensure that the perturbations remain statistically consistent with the observation model. We analyse how such adversarial yet plausible observations shift inference about latent states and affect downstream decision-making and the performance of reinforcement learning agents. In addition, we introduce an online Bayesian defence based on directional covariance adaptation, which selectively reduces the influence of observations by comparing their estimated impact with that of the computed most disruptive direction, while preserving information in the remaining orthogonal observation subspace. The proposed framework provides a principled approach to constructing robust inference and decision-making systems, with direct relevance to safety-critical applications such as robotics, where reliable operation under sensor noise, partial failures, and adversarial conditions is essential.