# OracleSage: Towards Unified Visual-Linguistic Understanding of Oracle Bone Scripts through Cross-Modal Knowledge Fusion

arXiv: https://arxiv.org/abs/2411.17837 · HTML: https://mlsanity.com/paper/2411.17837/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks](https://mlsanity.com/paper/2208.10442/) (2022)
- [Pixtral 12B](https://mlsanity.com/paper/2410.07073/) (2024)
- [An open dataset for oracle bone script recognition and decipherment](https://mlsanity.com/paper/2401.15365/) (2024)
- [An open dataset for the evolution of oracle bone characters: EVOBC](https://mlsanity.com/paper/2401.12467/) (2024)
- [Oracle Bone Inscriptions Multi-modal Dataset](https://mlsanity.com/paper/2407.03900/) (2024)
- [Diff-Oracle: Deciphering Oracle Bone Scripts with Controllable Diffusion Model](https://mlsanity.com/paper/2312.13631/) (2023)

## Built on by (most cited first, 10 of 10)

- [Opportunities and Challenges of Large Language Models for Low-Resource Languages in Humanities Research](https://mlsanity.com/paper/2412.04497/) (2024)
- [Foundation Models for Low-Resource Language Education (Vision Paper)](https://mlsanity.com/paper/2412.04774/) (2024)
- [Mitigating Long-tail Distribution in Oracle Bone Inscriptions: Dataset, Model, and Benchmark](https://mlsanity.com/paper/2504.09555/) (2025)
- [PictOBI-20k: Unveiling Large Multimodal Models in Visual Decipherment for Pictographic Oracle Bone Characters](https://mlsanity.com/paper/2509.05773/) (2025)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [Interpretable Oracle Bone Script Decipherment through Radical and Pictographic Analysis with LVLMs](https://mlsanity.com/paper/2508.10113/) (2025)
- [Towards Computational Chinese Paleography](https://mlsanity.com/paper/2601.06753/) (2026)
- [OracleAnalyser: Analysing Implicit Semantics of Oracle Bone Scripts through MLLMs with Post-training](https://mlsanity.com/paper/2606.25906/) (2026)
- [Beyond Single Character: Evaluating MLLMs for Sentence-Level Oracle Bone Inscription Understanding](https://mlsanity.com/paper/2606.31169/) (2026)
- [JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis](https://mlsanity.com/paper/2608.11741/) (2026)

## Abstract (arXiv)

Oracle bone script (OBS), as China's earliest mature writing system, present significant challenges in automatic recognition due to their complex pictographic structures and divergence from modern Chinese characters. We introduce OracleSage, a novel cross-modal framework that integrates hierarchical visual understanding with graph-based semantic reasoning. Specifically, we propose (1) a Hierarchical Visual-Semantic Understanding module that enables multi-granularity feature extraction through progressive fine-tuning of LLaVA's visual backbone, (2) a Graph-based Semantic Reasoning Framework that captures relationships between visual components and semantic concepts through dynamic message passing, and (3) OracleSem, a semantically enriched OBS dataset with comprehensive pictographic and semantic annotations. Experimental results demonstrate that OracleSage significantly outperforms state-of-the-art vision-language models. This research establishes a new paradigm for ancient text interpretation while providing valuable technical support for archaeological studies.