# Multimodal 3D Genome Pre-training

arXiv: https://arxiv.org/abs/2504.09060 · HTML: https://mlsanity.com/paper/2504.09060/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 58 references that are in the map.
- First posted on arXiv 12 Apr 2025.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day](https://mlsanity.com/paper/2306.00890/) (2023)
- [HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution](https://mlsanity.com/paper/2306.15794/) (2023)
- [Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling](https://mlsanity.com/paper/2307.14126/) (2023)
- [Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling](https://mlsanity.com/paper/2403.03234/) (2024)
- [CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models](https://mlsanity.com/paper/2406.06007/) (2024)
- [SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalization](https://mlsanity.com/paper/2310.19795/) (2023)
- [On the Importance of Asymmetry for Siamese Representation Learning](https://mlsanity.com/paper/2204.00613/) (2022)
- [Fundamental Limits and Tradeoffs in Invariant Representation Learning](https://mlsanity.com/paper/2012.10713/) (2020)
- [Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model](https://mlsanity.com/paper/2211.10590/) (2022)
- [Uni-Mol2: Exploring Molecular Pretraining Model at Scale](https://mlsanity.com/paper/2406.14969/) (2024)
- [Missing Modality Prediction for Unpaired Multimodal Learning via Joint Embedding of Unimodal Models](https://mlsanity.com/paper/2407.12616/) (2024)
- [UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning](https://mlsanity.com/paper/2405.10343/) (2024)
- [VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling](https://mlsanity.com/paper/2405.10812/) (2024)
- [Training Compute-Optimal Protein Language Models](https://mlsanity.com/paper/2411.02142/) (2024)
- [Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains](https://mlsanity.com/paper/2402.01481/) (2024)

## Abstract (arXiv)

Deep learning techniques have driven significant progress in various analytical tasks within 3D genomics in computational biology. However, a holistic understanding of 3D genomics knowledge remains underexplored. Here, we propose MIX-HIC, the first multimodal foundation model of 3D genome that integrates both 3D genome structure and epigenomic tracks, which obtains unified and comprehensive semantics. For accurate heterogeneous semantic fusion, we design the cross-modal interaction and mapping blocks for robust unified representation, yielding the accurate aggregation of 3D genome knowledge. Besides, we introduce the first large-scale dataset comprising over 1 million pairwise samples of Hi-C contact maps and epigenomic tracks for high-quality pre-training, enabling the exploration of functional implications in 3D genomics. Extensive experiments show that MIX-HIC can significantly surpass existing state-of-the-art methods in diverse downstream tasks. This work provides a valuable resource for advancing 3D genomics research.