# Bridging Vision and Language Concepts through Optimal Transport Semantic Flow

arXiv: https://arxiv.org/abs/2606.26891 · HTML: https://mlsanity.com/paper/2606.26891/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 25 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification](https://mlsanity.com/paper/2211.11158/) (2022)
- [Post-hoc Concept Bottleneck Models](https://mlsanity.com/paper/2205.15480/) (2022)
- [Label-Free Concept Bottleneck Models](https://mlsanity.com/paper/2304.06129/) (2023)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [Unbalanced minibatch Optimal Transport; applications to Domain Adaptation](https://mlsanity.com/paper/2103.03606/) (2021)
- [VLG-CBM: Training Concept Bottleneck Models with Vision-Language Guidance](https://mlsanity.com/paper/2408.01432/) (2024)
- [Unbalanced Optimal Transport, from Theory to Numerics](https://mlsanity.com/paper/2211.08775/) (2022)
- [Incremental Residual Concept Bottleneck Models](https://mlsanity.com/paper/2404.08978/) (2024)
- [Scaling Algorithms for Unbalanced Transport Problems](https://mlsanity.com/paper/1607.05816/) (2016)
- [Auxiliary Losses for Learning Generalizable Concept-based Models](https://mlsanity.com/paper/2311.11108/) (2023)
- [Flowing from Words to Pixels: A Noise-Free Framework for Cross-Modality Evolution](https://mlsanity.com/paper/2412.15213/) (2024)
- [Learning to Match via Inverse Optimal Transport](https://mlsanity.com/paper/1802.03644/) (2018)
- [Unbalanced Optimal Transport: A Unified Framework for Object Detection](https://mlsanity.com/paper/2307.02402/) (2023)
- [Stochastic bridges of linear systems](https://mlsanity.com/paper/1407.3421/) (2014)
- [Discovering Fine-Grained Visual-Concept Relations by Disentangled Optimal Transport Concept Bottleneck Models](https://mlsanity.com/paper/2505.07209/) (2025)
- [Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning](https://mlsanity.com/paper/2404.03323/) (2024)
- [Discrete Probabilistic Inverse Optimal Transport](https://mlsanity.com/paper/2112.09754/) (2021)
- [OAT-FM: Optimal Acceleration Transport for Improved Flow Matching](https://mlsanity.com/paper/2509.24936/) (2025)
- [Monge-Kantorovich Optimal Transport Through Constrictions and Flow-rate Constraints](https://mlsanity.com/paper/2212.14509/) (2022)
- [Data Assimilation for Sign-indefinite Priors: A generalization of Sinkhorn's algorithm](https://mlsanity.com/paper/2308.11791/) (2023)
- [Network Learning with Directional Sign Patterns](https://mlsanity.com/paper/2403.14915/) (2024)

## Built on by (most cited first, 1 of 1)

- [Readout Blindness: VLM Scores Miss the Spatial Direction Their Frozen Encoders Retain](https://mlsanity.com/paper/2610.06324/) (2026)

## Abstract (arXiv)

Concept Bottleneck Models (CBMs) promise transparent reasoning by predicting through human-interpretable concepts, yet their effectiveness fundamentally depends on how well visual and textual representations are aligned or matched. Existing vision-language CBMs often rely on pre-aligned encoders or global cosine similarity, which obscures fine-grained concept localization and fails to reflect true semantic geometry. In this work, we rethink concept alignment as a dynamic cross-modal transport process instead of static projection and propose the Optimal Transport Flow Concept Bottleneck Model (OTF-CBM). It first learns a data-driven semantic cost via Inverse Optimal Transport to measure cross-modal distances, and then performs unbalanced optimal-transport-based flow matching to model semantic transitions between visual patches and textual concepts. With velocity-based concept activation, OTF-CBM captures interpretable geometric relations without ODE integration. Experiments further show that OTF-CBM achieves superior classification accuracy and concept faithfulness, offering a new geometric and dynamical perspective for interpretable cross-modal reasoning.