# Self-Labeling of Fully Mediating Representations by Graph Alignment

arXiv: https://arxiv.org/abs/2103.14133 · HTML: https://mlsanity.com/paper/2103.14133/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 45 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 9 of 9)

- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [A review of domain adaptation without target labels](https://mlsanity.com/paper/1901.05335/) (2019)
- [Molecular Structure Extraction From Documents Using Deep Learning](https://mlsanity.com/paper/1802.04903/) (2018)
- [ChemGrapher: Optical Graph Recognition of Chemical Compounds by Deep Learning](https://mlsanity.com/paper/2002.09914/) (2020)
- [Image-Conditioned Graph Generation for Road Network Extraction](https://mlsanity.com/paper/1910.14388/) (2019)
- [Road Segmentation in SAR Satellite Images with Deep Fully-Convolutional Neural Networks](https://mlsanity.com/paper/1802.01445/) (2018)

## Built on by (most cited first, 3 of 3)

- [Atom-Level Optical Chemical Structure Recognition with Limited Supervision](https://mlsanity.com/paper/2404.01743/) (2024)
- [Weakly Supervised Knowledge Transfer with Probabilistic Logical Reasoning for Object Detection](https://mlsanity.com/paper/2303.05148/) (2023)
- [ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding](https://mlsanity.com/paper/2605.17214/) (2026)

## Abstract (arXiv)

To be able to predict a molecular graph structure (W) given a 2D image of a chemical compound (U) is a challenging problem in machine learning. We are interested to learn f: U rightarrow W where we have a fully mediating representation V such that f factors into U rightarrow V rightarrow W. However, observing V requires detailed and expensive labels. We propose graph aligning approach that generates rich or detailed labels given normal labels W. In this paper we investigate the scenario of domain adaptation from the source domain where we have access to the expensive labels V to the target domain where only normal labels W are available. Focusing on the problem of predicting chemical compound graphs from 2D images the fully mediating layer is represented using the planar embedding of the chemical graph structure we are predicting. The use of a fully mediating layer implies some assumptions on the mechanism of the underlying process. However if the assumptions are correct it should allow the machine learning model to be more interpretable, generalize better and be more data efficient at training time. The empirical results show that, using only 4000 data points, we obtain up to 4x improvement of performance after domain adaptation to target domain compared to pretrained model only on the source domain. After domain adaptation, the model is even able to detect atom types that were never seen in the original source domain. Finally, on the Maybridge data set the proposed self-labeling approach reached higher performance than the current state of the art.