mlsanityData to 14 Oct 2026

Contrastive-to-Self-Supervised: A Two-Stage Framework for Script Similarity Learning

arXiv 2603.06180 ↗First posted 6 Mar 2026cs.CV cs.AI cs.CLcited by 1

On the map: Knowledge distillation → Knowledge distillation for LLMs

Abstract (arXiv)

Learning similarity metrics for glyphs and writing systems faces a fundamental challenge: while individual graphemes within invented alphabets can be reliably labeled, the historical relationships between different scripts remain uncertain and contested. We propose a two-stage framework that addresses this epistemological constraint. First, we train an encoder with contrastive loss on labeled invented alphabets, establishing a teacher model with robust discriminative features. Second, we extend to historically attested scripts through teacher-student distillation, where the student learns unsupervised representations guided by the teacher's knowledge but free to discover latent cross-script similarities. The asymmetric setup enables the student to learn deformation-invariant embeddings while inheriting discriminative structure from clean examples. Our approach bridges supervised contrastive learning and unsupervised discovery, enabling both hard boundaries between distinct systems and soft similarities reflecting potential historical influences. Experiments on diverse writing systems demonstrate effective few-shot glyph recognition and meaningful script clustering without requiring ground-truth evolutionary relationships.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Knowledge distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 13 of its 25 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

13 of 25 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  10. 2016Matching Networks for One Shot Learningcited by 2,255
  11. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  12. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
1 more
  1. 2019The Omniglot challenge: a 3-year progress reportcited by 54

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Language-Agnostic Visual Embeddings for Cross-Script Handwriting Retrieval
  2. 2025Reasoning Over the Glyphs: Evaluation of LLM's Decipherment of Rare Scripts
  3. 2026One Language, Two Scripts: Probing Script-Invariance in LLM Concept Representations
  4. 2024mathbbX-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs
  5. 2025GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing
  6. 2024GlyphPattern: An Abstract Pattern Recognition Benchmark for Vision-Language Models
  7. 2022Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation
  8. 2026Art Beyond Semantics: Sheaf-Informed Contrastive Learning for Multi-Relational Representations