# Syn-Omni: Structured Specialization and Progressive Collaboration for Omnimodal Embeddings

arXiv: https://arxiv.org/abs/2610.12256 · HTML: https://mlsanity.com/paper/2610.12256/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 102 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 12 of 12)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [E5-V: Universal Embeddings with Multimodal Large Language Models](https://mlsanity.com/paper/2407.12580/) (2024)
- [Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities](https://mlsanity.com/paper/2505.17862/) (2025)
- [MoELoRA: Contrastive Learning Guided Mixture of Experts on Parameter-Efficient Fine-Tuning for Large Language Models](https://mlsanity.com/paper/2402.12851/) (2024)
- [Omni-Embed-Nemotron: A Unified Multimodal Retrieval Model for Text, Image, Audio, and Video](https://mlsanity.com/paper/2510.03458/) (2025)
- [Towards Universal Video Retrieval: Generalizing Video Embedding via Synthesized Multimodal Pyramid Curriculum](https://mlsanity.com/paper/2510.27571/) (2025)
- [MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval](https://mlsanity.com/paper/2510.15543/) (2025)
- [Scaling Audio-Text Retrieval with Multimodal Large Language Models](https://mlsanity.com/paper/2602.18010/) (2026)
- [MAGMaR Shared Task System Description: Video Retrieval with OmniEmbed](https://mlsanity.com/paper/2506.09409/) (2025)

## Abstract (arXiv)

Omnimodal embeddings naturally involve both shared representations and modality-specific features across heterogeneous inputs. However, existing omnimodal embedding methods often rely on a single shared parameter space over mixed-modality data, limiting structural separation between universal and modality-specific representations. To address this, we propose Syn-Omni, a unified framework for structured omnimodal adaptation with modality specialization and controlled cross-modal collaboration. Specifically, we introduce Orthogonal Modality-Expert LoRA (OME-LoRA), which decomposes adaptation into a shared LoRA path for universal semantics and modality-expert LoRA paths for modality-aware specialization. Furthermore, Progressive Synergy Routing (PSR) enables experts to first establish modality-specific priors, then gradually interact with other modality-experts for cross-modal synergy. Evaluated across 81 diverse tasks spanning image, video, audio, and audiovisual modalities, Syn-Omni consistently outperforms omnimodal baselines, demonstrating the effectiveness of structured specialization and cross-modal progressive collaboration.