Contrastive-to-Self-Supervised: A Two-Stage Framework for Script Similarity Learning
On the map: Knowledge distillation → Knowledge distillation for LLMs
Abstract (arXiv)
Learning similarity metrics for glyphs and writing systems faces a fundamental challenge: while individual graphemes within invented alphabets can be reliably labeled, the historical relationships between different scripts remain uncertain and contested. We propose a two-stage framework that addresses this epistemological constraint. First, we train an encoder with contrastive loss on labeled invented alphabets, establishing a teacher model with robust discriminative features. Second, we extend to historically attested scripts through teacher-student distillation, where the student learns unsupervised representations guided by the teacher's knowledge but free to discover latent cross-script similarities. The asymmetric setup enables the student to learn deformation-invariant embeddings while inheriting discriminative structure from clean examples. Our approach bridges supervised contrastive learning and unsupervised discovery, enabling both hard boundaries between distinct systems and soft similarities reflecting potential historical influences. Experiments on diverse writing systems demonstrate effective few-shot glyph recognition and meaningful script clustering without requiring ground-truth evolutionary relationships.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 13 of its 25 references that are in the map. First posted on arXiv 6 Mar 2026.
Builds on
13 of 25 references.
References · 13
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2017Prototypical Networks for Few-shot Learningcited by 3,015
- 2020Supervised Contrastive Learningcited by 2,829
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
- 2016Matching Networks for One Shot Learningcited by 2,255
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
1 more
- 2019The Omniglot challenge: a 3-year progress reportcited by 54
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Language-Agnostic Visual Embeddings for Cross-Script Handwriting Retrieval
- 2025Reasoning Over the Glyphs: Evaluation of LLM's Decipherment of Rare Scripts
- 2026One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations
- 2024mathbbX-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs
- 2025GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing
- 2024GlyphPattern: An Abstract Pattern Recognition Benchmark for Vision-Language Models
- 2022Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation
- 2026Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representations