mlsanityData to 14 Oct 2026

Critic Sequential Monte Carlo

arXiv 2205.15460 ↗First posted 30 May 2022stat.ML cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

We introduce CriticSMC, a new algorithm for planning as inference built from a composition of sequential Monte Carlo with learned Soft-Q function heuristic factors. These heuristic factors, obtained from parametric approximations of the marginal likelihood ahead, more effectively guide SMC towards the desired target distribution, which is particularly helpful for planning in environments with hard constraints placed sparsely in time. Compared with previous work, we modify the placement of such heuristic factors, which allows us to cheaply propose and evaluate large numbers of putative action particles, greatly increasing inference and planning efficiency. CriticSMC is compatible with informative priors, whose density function need not be known, and can be used as a model-free control algorithm. Our experiments on collision avoidance in a high-dimensional simulated driving task show that CriticSMC significantly reduces collision rates at a low computational cost while maintaining realism and diversity of driving behaviors across vehicles and environment scenarios.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 41 of its 88 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

41 of 88 references.

References · 41

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2013Auto-Encoding Variational Bayescited by 4,053
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2016OpenAI Gymcited by 1,736
  7. 2018World Modelscited by 1,513
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  9. 2015Prioritized Experience Replaycited by 1,099
  10. 2016End to End Learning for Self-Driving Carscited by 1,009
  11. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  12. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
29 more
  1. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  2. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  3. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  4. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  5. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 340
  6. 2018Maximum a Posteriori Policy Optimisationcited by 328
  7. 2019INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Mapscited by 231
  8. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 206
  9. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 193
  10. 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
  11. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  12. 2020SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learningcited by 152
  13. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  14. 2019Urban Driving with Conditional Imitation Learningcited by 86
  15. 2019V-MPO: On-Policy Maximum a Posteriori Policy Optimization for Discrete and Continuous Controlcited by 85
  16. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  17. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  18. 2020Learning Off-Policy with Online Planningcited by 53
  19. 2018DiCE: The Infinitely Differentiable Monte-Carlo Estimatorcited by 46
  20. 2021Greedification Operators for Policy Optimization: Investigating Forward and Reverse KL Divergencescited by 41
  21. 2015Neural Adaptive Sequential Monte Carlocited by 34
  22. 2021Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulationcited by 32
  23. 2021Autonomy 2.0: Why is self-driving always 5 years away?cited by 30
  24. 2020Learning to Locomote: Understanding How Environment Design Matters for Deep Reinforcement Learningcited by 27
  25. 2019CAQL: Continuous Action Q-Learningcited by 23
  26. 2019DualSMC: Tunneling Differentiable Filtering and Planning under Continuous POMDPscited by 21
  27. 2021Reimagining an autonomous vehiclecited by 11
  28. 2014Sequential Monte Carlo for Graphical Modelscited by 4
  29. 2015Coarse-to-Fine Sequential Monte Carlo for Probabilistic Programscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 1; 2024: 3; 2025: 3; 2026: 7.

16 papers.

Sort:

By area:

  1. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 129
  2. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  3. 2026On the Power of (Approximate) Reward Models for Inference-Time Scalingcited by 7
  4. 2022Improved Policy Optimization for Online Imitation Learningcited by 6
  5. 2025Trust-Region Twisted Policy Improvementcited by 6
  6. 2024SPO: Sequential Monte Carlo Policy Optimisationcited by 5
  7. 2026Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusioncited by 5
  8. 2023NAS-X: Neural Adaptive Smoothing via Twistingcited by 4
  9. 2025Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scalingcited by 4
  10. 2026When are likely answers right? On Sequence Probability and Correctness in LLMscited by 4
  11. 2025Twice Sequential Monte Carlo for Tree Searchcited by 3
  12. 2026Improved Constrained Generation by Bridging Pretrained Generative Modelscited by 2
4 more
  1. 2024Robust Inverse Graphics via Probabilistic Inferencecited by 1
  2. 2026VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learningcited by 1
  3. 2026Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agentscited by 1
  4. 2026Integration Matters: Rollout-Based Training for Constrained Diffusion Modelscited by 0