mlsanityData to 13 Oct 2026

Epistemic Uncertainty Quantification for Pre-trained VLMs via Riemannian Flow Matching

arXiv 2601.21662 ↗First posted 29 Jan 2026cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

also in 1 more
Abstract (arXiv)

Vision-Language Models (VLMs) are typically deterministic in nature and lack intrinsic mechanisms to quantify epistemic uncertainty, which reflects the model's lack of knowledge or ignorance of its own representations. We theoretically motivate negative log-density of an embedding as a proxy for the epistemic uncertainty, where low-density regions signify model ignorance. The proposed method REPVLM computes the probability density on the hyperspherical manifold of the VLM embeddings using Riemannian Flow Matching. We empirically demonstrate that REPVLM achieves near-perfect correlation between uncertainty and prediction error, significantly outperforming existing baselines. Beyond classification, we also demonstrate that the model also provides a scalable metric for out-of-distribution detection and automated data curation.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 50 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

34 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere16 May
  2. 2025Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable Models8 May
  3. 2024Post-hoc Probabilistic Vision-Language Models8 Dec
  4. 2024Probabilistic Language-Image Pre-Training24 Oct

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2022Flow Matching for Generative Modelingcited by 5,542
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,430
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,314
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,141
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,902
  7. 2018Neural Ordinary Differential Equationscited by 3,750
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,068
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,397
  10. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,164
  11. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,012
  12. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 910
18 more
  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 837
  2. 2017Selective Classification for Deep Neural Networkscited by 825
  3. 2021Robust fine-tuning of zero-shot modelscited by 790
  4. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 755
  5. 2023DataComp: In search of the next generation of multimodal datasetscited by 639
  6. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 588
  7. 2018Do Better ImageNet Models Transfer Better?cited by 562
  8. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 555
  9. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  10. 2024The Platonic Representation Hypothesiscited by 369
  11. 2023Flow Matching on General Geometriescited by 262
  12. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  13. 2021Probabilistic Embeddings for Cross-Modal Retrievalcited by 150
  14. 2024The Double-Ellipsoid Geometry of CLIPcited by 42
  15. 2023Improved Probabilistic Image-Text Representationscited by 40
  16. 2025Whitened CLIP as a Likelihood Surrogate of Images and Captionscited by 16
  17. 2025On the Importance of Embedding Norms in Self-Supervised Learningcited by 9
  18. 2026The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergencecited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Uncertainty Quantification for Flow-Based Vision-Language-Action Modelscited by 4
  2. 2026GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embeddingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Art of Saying "Maybe": A Conformal Lens for Uncertainty Benchmarking in VLMs
  2. 2026FUSE: Quantifying Uncertainty in Vision-Language Models by Bayesian Fusing Epistemic and Aleatoric Uncertainty
  3. 2026Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
  4. 2026Dynamic Distribution-Aware Uncertainty Tracking in Vision-Language Representation Learning
  5. 2026VLM-UQBench: A Benchmark for Modality-Specific and Cross-Modality Uncertainties in Vision Language Models
  6. 2026VAUQ: Vision-Aware Uncertainty Quantification for LVLM Self-Evaluation
  7. 2024Uncertainty-Aware Evaluation for Vision-Language Models
  8. 2025Intra-Class Probabilistic Embeddings for Uncertainty Estimation in Vision-Language Models