# The Hybrid Multimodal Graph Index (HMGI): A Comprehensive Framework for Integrated Relational and Vector Search

arXiv: https://arxiv.org/abs/2510.10123 · HTML: https://mlsanity.com/paper/2510.10123/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 62 references that are in the map.
- First posted on arXiv 11 Oct 2025.

## Builds on (most cited first, 23 of 23)

- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [On the Theoretical Limitations of Embedding-Based Retrieval](https://mlsanity.com/paper/2508.21038/) (2025)
- [A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2101.12631/) (2021)
- [A Comprehensive Survey on Vector Database: Storage and Retrieval Technique, Challenge](https://mlsanity.com/paper/2310.11703/) (2023)
- [MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attention](https://mlsanity.com/paper/2504.16083/) (2025)
- [Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch](https://mlsanity.com/paper/2406.00552/) (2024)
- [Vector database management systems: Fundamental concepts, use-cases, and current challenges](https://mlsanity.com/paper/2309.11322/) (2023)
- [Effective and General Distance Computation for Approximate Nearest Neighbor Search](https://mlsanity.com/paper/2404.16322/) (2024)
- [Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query Expansion](https://mlsanity.com/paper/2506.04760/) (2025)
- [TRACE: Transparent Web Reliability Assessment with Contextual Explanations](https://mlsanity.com/paper/2506.12072/) (2025)
- [NaviX: A Native Vector Index Design for Graph DBMSs With Robust Predicate-Agnostic Search Performance](https://mlsanity.com/paper/2506.23397/) (2025)
- [Domain-Enhanced Dual-Branch Model for Efficient and Interpretable Accident Anticipation](https://mlsanity.com/paper/2507.12755/) (2025)
- [DualCast: A Model to Disentangle Aperiodic Events from Traffic Series](https://mlsanity.com/paper/2411.18286/) (2024)
- [DEG: Efficient Hybrid Vector Search Using the Dynamic Edge Navigation Graph](https://mlsanity.com/paper/2502.07343/) (2025)
- [A Topology-Aware Localized Update Strategy for Graph-Based ANN Index](https://mlsanity.com/paper/2503.00402/) (2025)
- [LIRA: A Learning-based Query-aware Partition Framework for Large-scale ANN Search](https://mlsanity.com/paper/2503.23409/) (2025)
- [Aura-CAPTCHA: A Reinforcement Learning and GAN-Enhanced Multi-Modal CAPTCHA System](https://mlsanity.com/paper/2508.14976/) (2025)
- [On Embeddings in Relational Databases](https://mlsanity.com/paper/2005.06437/) (2020)
- [Breaking the Storage-Compute Bottleneck in Billion-Scale ANNS: A GPU-Driven Asynchronous I/O Framework](https://mlsanity.com/paper/2507.10070/) (2025)
- [Vector embedding of multi-modal texts: a tool for discovery?](https://mlsanity.com/paper/2509.08216/) (2025)
- [Looking Back at Postgres](https://mlsanity.com/paper/1901.01973/) (2019)
- [A Generative Framework for Predictive Modeling of Multiple Chronic Conditions Using Graph Variational Autoencoder and Bandit-Optimized Graph Neural Network](https://mlsanity.com/paper/2409.13671/) (2024)
- [Complementarity-driven Representation Learning for Multi-modal Knowledge Graph Completion](https://mlsanity.com/paper/2507.20620/) (2025)

## Abstract (arXiv)

The proliferation of complex, multimodal datasets has exposed a critical gap between the capabilities of specialized vector databases and traditional graph databases. While vector databases excel at semantic similarity search, they lack the capacity for deep relational querying. Conversely, graph databases master complex traversals but are not natively optimized for high-dimensional vector search. This paper introduces the Hybrid Multimodal Graph Index (HMGI), a novel framework designed to bridge this gap by creating a unified system for efficient, hybrid queries on multimodal data. HMGI leverages the native graph database architecture and integrated vector search capabilities, exemplified by platforms like Neo4j, to combine Approximate Nearest Neighbor Search (ANNS) with expressive graph traversal queries. Key innovations of the HMGI framework include modality-aware partitioning of embeddings to optimize index structure and query performance, and a system for adaptive, low-overhead index updates to support dynamic data ingestion, drawing inspiration from the architectural principles of systems like TigerVector. By integrating semantic similarity search directly with relational context, HMGI aims to outperform pure vector databases like Milvus in complex, relationship-heavy query scenarios and achieve sub-linear query times for hybrid tasks.