# RiVaT-Fuse: Reliability-Calibrated Variational Tensor Fusion with Matrix-Valued Trust for Multimodal Prediction under Modality Uncertainty

arXiv: https://arxiv.org/abs/2609.10798 · HTML: https://mlsanity.com/paper/2609.10798/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 55 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 5 of 5)

- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Embedding Foundation Model Predictions in Discrete-Choice Models with Structural Guarantees](https://mlsanity.com/paper/2606.26432/) (2026)
- [OSEF: One-Step Evidence Fusion for Cross-Video Scene Procedure Planning](https://mlsanity.com/paper/2607.29401/) (2026)
- [Learning When to Trust via Selective Context Preference Optimization](https://mlsanity.com/paper/2608.06377/) (2026)
- [VOICE: A Vision-Omics Foundation Model Integrating Direct and Retrieval-Based Prediction of In-situ Single-Cell Gene Expression](https://mlsanity.com/paper/2608.08366/) (2026)

## Built on by (most cited first, 2 of 2)

- [Simple Agentic Memory for Generalist Robot Policies](https://mlsanity.com/paper/2609.36595/) (2026)
- [When Can Text Replace Vision? Structural Bottlenecks in Diagram Reasoning](https://mlsanity.com/paper/2609.39142/) (2026)

## Abstract (arXiv)

Multimodal prediction from images and structured metadata requires integrating complementary evidence whose reliability can vary across samples and latent directions. A single confidence weight per modality cannot capture this directional variation. We propose RiVaT-Fuse, a reliability-calibrated variational tensor fusion framework that formulates fusion as sample-wise latent-state estimation. For each image-metadata pair, the fused representation minimizes a quadratic objective combining agreement with modality embeddings, structured cross-modal interactions, and regularization. Positive-definite, low-rank-plus-diagonal trust matrices are conditioned on learned state descriptors and metadata completeness, allowing modality contributions to vary across latent directions. Additive, multiplicative, and relational interactions model cross-modal dependencies within the latent estimation objective. The resulting system admits a unique solution computed through a differentiable linear solve. A first-order analysis with fixed trust operators relates latent sensitivity to system conditioning and perturbations in modality embeddings and interactions. The framework further incorporates a state-binned entropic surrogate for conditional distributionally robust learning and task-coupled quadratic prediction heads. We instantiate RiVaT-Fuse on mBRSET, pairing retinal images with clinical and demographic metadata for diabetic retinopathy grading, diabetic macular edema detection, and referable-status prediction. Comparisons with unimodal and representation-level fusion baselines assess the predictive utility of the complete framework across these related clinical tasks.