# Discriminative Particle Filter Reinforcement Learning for Complex Partial Observations

arXiv: https://arxiv.org/abs/2002.09884 · HTML: https://mlsanity.com/paper/2002.09884/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2020.

## Built on by (most cited first, 28 of 28)

- [Differentiable Particle Filtering via Entropy-Regularized Optimal Transport](https://mlsanity.com/paper/2102.07850/) (2021)
- [Deep Transformer Q-Networks for Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2206.01078/) (2022)
- [Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation](https://mlsanity.com/paper/2105.07593/) (2021)
- [BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark](https://mlsanity.com/paper/2407.07788/) (2024)
- [An overview of differentiable particle filters for data-adaptive sequential Bayesian inference](https://mlsanity.com/paper/2302.09639/) (2023)
- [Recurrent Off-policy Baselines for Memory-based Continuous Control](https://mlsanity.com/paper/2110.12628/) (2021)
- [Online Variational Filtering and Parameter Learning](https://mlsanity.com/paper/2110.13549/) (2021)
- [Leveraging Fully Observable Policies for Learning under Partial Observability](https://mlsanity.com/paper/2211.01991/) (2022)
- [Equivariant Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2408.14336/) (2024)
- [Informed POMDP: Leveraging Additional Information in Model-Based RL](https://mlsanity.com/paper/2306.11488/) (2023)
- [Normalizing Flow-based Differentiable Particle Filters](https://mlsanity.com/paper/2403.01499/) (2024)
- [SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2105.06228/) (2021)
- [Variational Marginal Particle Filters](https://mlsanity.com/paper/2109.15134/) (2021)
- [Beyond Local Views: Global State Inference with Diffusion Models for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2408.09501/) (2024)
- [Sequential Monte Carlo for Policy Optimization in Continuous POMDPs](https://mlsanity.com/paper/2505.16732/) (2025)
- [Particle-Based Score Estimation for State Space Model Learning in Autonomous Driving](https://mlsanity.com/paper/2212.06968/) (2022)
- [On-Robot Bayesian Reinforcement Learning for POMDPs](https://mlsanity.com/paper/2307.11954/) (2023)
- [PF-GNN: Differentiable particle filtering based approximation of universal graph representations](https://mlsanity.com/paper/2401.17752/) (2024)
- [Model-Based Reinforcement Learning under Random Observation Delays](https://mlsanity.com/paper/2509.20869/) (2025)
- [Learning to Navigate in Mazes with Novel Layouts using Abstract Top-down Maps](https://mlsanity.com/paper/2412.12024/) (2024)
- [Deep Reinforcement Learning via Object-Centric Attention](https://mlsanity.com/paper/2504.03024/) (2025)
- [BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs](https://mlsanity.com/paper/2202.08884/) (2022)
- [Missing Data Multiple Imputation for Tabular Q-Learning in Online RL](https://mlsanity.com/paper/2510.10709/) (2025)
- [ESCORT: Efficient Stein-variational and Sliced Consistency-Optimized Temporal Belief Representation for POMDPs](https://mlsanity.com/paper/2510.21107/) (2025)
- [Perception-Based Beliefs for POMDPs with Visual Observations](https://mlsanity.com/paper/2602.05679/) (2026)
- [Universal Approximation and the Topological Neural Network](https://mlsanity.com/paper/2305.16639/) (2023)
- [Interaction-Aware Parameter Privacy-Preserving Data Sharing in Coupled Systems via Particle Filter Reinforcement Learning](https://mlsanity.com/paper/2505.06122/) (2025)
- [Reinforcement Learning for Heterogeneous Sensor Selection in Maritime Surveillance](https://mlsanity.com/paper/2607.22667/) (2026)

## Abstract (arXiv)

Deep reinforcement learning is successful in decision making for sophisticated games, such as Atari, Go, etc. However, real-world decision making often requires reasoning with partial information extracted from complex visual observations. This paper presents Discriminative Particle Filter Reinforcement Learning (DPFRL), a new reinforcement learning framework for complex partial observations. DPFRL encodes a differentiable particle filter in the neural network policy for explicit reasoning with partial observations over time. The particle filter maintains a belief using learned discriminative update, which is trained end-to-end for decision making. We show that using the discriminative update instead of standard generative models results in significantly improved performance, especially for tasks with complex visual observations, because they circumvent the difficulty of modeling complex observations that are irrelevant to decision making. In addition, to extract features from the particle belief, we propose a new type of belief feature based on the moment generating function. DPFRL outperforms state-of-the-art POMDP RL models in Flickering Atari Games, an existing POMDP RL benchmark, and in Natural Flickering Atari Games, a new, more challenging POMDP RL benchmark introduced in this paper. Further, DPFRL performs well for visual navigation with real-world data in the Habitat environment.