Symmetry-Aware Likelihood-Orbit Aggregation for Selective Left-Right Claim Verification
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Frozen vision-language models (VLMs) remain unreliable on fine-grained left-right claims, and raw claim likelihoods need not reliably rank verification errors. After a horizontal-reflection intervention is fixed, how should its induced likelihood measurements be combined into a selective verification signal? We introduce Relation-Orbit, a closed-form contrast with no learned fusion parameters that assigns eight normalized likelihoods to query-supporting and counterfactual roles determined by reflection, inverse relation, and entity exchange. A claim is asserted only when the signed contrast exceeds a threshold selected on held-out data using pointwise Clopper-Pearson upper confidence bounds. On VSR and GQA across four frozen VLMs, Relation-Orbit yields higher mean test coverage at a 10% selective-risk calibration target than an all-eight Orbit-Max baseline in all eight dataset-backbone settings; gains over a nearly abstain-all one-sided intervention score are reported separately. A separate LLaVA-1.5/COCO evaluation, reduced-orbit controls, and a two-sided partition diagnostic further characterize the structural advantage.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 23 references that are in the map. First posted on arXiv 15 Sep 2026.
Builds on
20 of 23 references.
Closest ancestors · 2
Same direction, newest first
- 2024Consistency and Uncertainty: Identifying Unreliable Responses From Black-Box Vision-Language Models for Selective Visual Question Answering16 Apr
- 2024Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning23 Feb
Other references · 18
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
- 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
- 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
- 2022Conformal Risk Controlcited by 299
- 2022Visual Spatial Reasoningcited by 293
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 254
6 more
- 2019SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognitioncited by 55
- 2026Predicting When to Trust Vision-Language Models for Spatial Reasoningcited by 5
- 2026Self-Evolving Spatial Reasoning in Vision Language Models via Geometric Logic Consistencycited by 3
- 2026Look Again Before You Abstain:Budgeted Conformal Evidence Acquisition for Reliable Vision-Language Modelcited by 3
- 2026SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classificationcited by 1
- 2026Grounding Isn't Knowing: Do VLMs Need Object Localization for Spatial Reasoning?cited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RECAP: Relation Evidence Calibration for Detecting Spatial Relation Hallucinations in Vision-Language Models
- 2026Left-Right Symmetry Breaking in CLIP-style Vision-Language Models Trained on Synthetic Spatial-Relation Data
- 2026COMPASS: Membership Inference Attack Against Vision Large Language Models via Adaptive Noise Calibration
- 2026VISTA: Auditing Semantic Divergence in Vision-Language Models
- 2026The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models
- 2026EviScope: Paired Counterfactual Evidence Diagnostics for Faithful and Efficient Grounded Language Models
- 2026Answer-Level Trust Selection for Physical Vision-Language Reasoning
- 2026SIFT: Robust Meta-Faithfulness Verification of Chain-of-Thought Reasoning Under Distribution Shift