mlsanityData to 7 Oct 2026

Conformal Prediction Intervals for Markov Decision Process Trajectories

arXiv 2206.04860 ↗First posted 10 Jun 2022cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

Before delegating a task to an autonomous system, a human operator may want a guarantee about the behavior of the system. This paper extends previous work on conformal prediction for functional data and conformalized quantile regression to provide conformal prediction intervals over the future behavior of an autonomous system executing a fixed control policy on a Markov Decision Process (MDP). The prediction intervals are constructed by applying conformal corrections to prediction intervals computed by quantile regression. The resulting intervals guarantee that with probability 1-δ the observed trajectory will lie inside the prediction interval, where the probability is computed with respect to the starting state distribution and the stochasticity of the MDP. The method is illustrated on MDPs for invasive species management and StarCraft2 battles.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 14 of its 33 references that are in the map. First posted on arXiv 10 Jun 2022.

Builds on

14 of 33 references.

References · 14

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  2. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  3. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  4. 2019Conformalized Quantile Regressioncited by 620
  5. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  6. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  7. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  8. 2020Explainable Reinforcement Learning: A Surveycited by 139
  9. 2013A Conformal Prediction Approach to Explore Functional Datacited by 85
  10. 2021Conformalized Survival Analysiscited by 53
  11. 2021Conformal Prediction Bands for Multivariate Functional Datacited by 28
  12. 2021Out-of-Distribution Dynamics Detection: RL-Relevant Benchmarks and Resultscited by 14
2 more
  1. 2021From Heatmaps to Structural Explanations of Image Classifierscited by 4
  2. 2021Function-on-function partial quantile regressioncited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 5; 2025: 3.

12 papers.

  1. 2023Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictionscited by 46
  2. 2024Conformalized Credal Set Predictorscited by 21
  3. 2023Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Controlcited by 19
  4. 2024Bellman Conformal Inference: Calibrating Prediction Intervals For Time Seriescited by 12
  5. 2024Conformalized Interactive Imitation Learning: Handling Expert Shift and Intermittent Feedbackcited by 11
  6. 2023Conformal Off-Policy Evaluation in Markov Decision Processescited by 6
  7. 2024Conformalized Teleoperation: Confidently Mapping Human Inputs to High-Dimensional Robot Actionscited by 4
  8. 2024Uncertainty quantification in metric spacescited by 3
  9. 2025Reliable uncertainty quantification for 2D/3D anatomical landmark localization using multi-output conformal predictioncited by 2
  10. 2025Conformal and kNN Predictive Uncertainty Quantification Algorithms in Metric Spacescited by 2
  11. 2025Conformal Prediction Beyond the Horizon: Distribution-Free Inference for Policy Evaluationcited by 2
  12. 2022Will My Robot Achieve My Goals? Predicting the Probability that an MDP Policy Reaches a User-Specified Behavior Targetcited by 0