mlsanityData to 7 Oct 2026

ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigation

arXiv 2609.17499 ↗First posted 15 Sep 2026cs.LG cs.AI cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

Uncertainty estimation for Vision-Language-Navigation (VLN) models is a critical task since it can help identify ambiguous and unreliable predictions, enabling agents to make safer navigation decisions. As one of the most advanced uncertainty estimation frameworks, conformal prediction (CP) offers a promising approach for uncertainty estimation in VLN. However, given that VLN agent requires a sequence of steps, standard calibration in conformal prediction fails to provide coverage guarantee it promises over a dependent, variable-length VLN episode. To this end, we propose Episode-Normalized Conformal Prediction (ENCP), which rescales a nonconformity score by the policy's residual confidence and calibrates one maximum score per episode. Under exchangeable calibration and test episodes, this construction covers the ground truth at every step with probability at least 1 - α, while allowing dependence among steps within an episode. Across four VLN policies and three nonconformity scores on R2R and REVERIE dataset, ENCP meets all reported empirical step-coverage targets on the seen-to-unseen evaluation. These results demonstrate that ENCP can provide model-agnostic uncertainty estimates, which might be useful for determining when a VLN agent should defer to a more capable predictor, including human assistance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

28 of 35 references.

References · 28

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2016Concrete Problems in AI Safetycited by 1,582
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  8. 2017Selective Classification for Deep Neural Networkscited by 799
  9. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  10. 2019Conformal Prediction Under Covariate Shiftcited by 511
  11. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  12. 2020Classification with Valid and Adaptive Coveragecited by 390
16 more
  1. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  2. 2022Conformal prediction beyond exchangeabilitycited by 348
  3. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  4. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  5. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  6. 2021Unsolved Problems in ML Safetycited by 291
  7. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  8. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  9. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  10. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  11. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  12. 2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 112
  13. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  14. 2022Adaptive Conformal Prediction for Motion Planning among Dynamic Agentscited by 46
  15. 2024Introspective Planning: Aligning Robots' Uncertainty with Inherent Task Ambiguitycited by 26
  16. 2024To Ask or Not to Ask? Detecting Absence of Information in Vision and Language Navigationcited by 1

Built on by

Not cited yet.