mlsanityData to 7 Oct 2026

R2-CoD: Understanding Text-Graph Complementarity in Relational Reasoning via Knowledge Co-Distillation

arXiv 2508.01475 ↗First posted 2 Aug 2025cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Relational reasoning lies at the core of many NLP tasks, drawing on complementary signals from text and graphs. While prior research has investigated how to leverage this dual complementarity, a detailed and systematic understanding of text-graph interplay and its effect on hybrid models remains underexplored. We take an analysis-driven approach to investigate text-graph representation complementarity via a unified architecture that supports knowledge co-distillation (CoD). We explore five tasks involving relational reasoning that differ in how text and graph structures encode the information needed to solve that task. By tracking how these dual representations evolve during training, we uncover interpretable patterns of alignment and divergence, and provide insights into when and why their integration is beneficial.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 60 references that are in the map. First posted on arXiv 2 Aug 2025.

Builds on

39 of 60 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2017Graph Attention Networkscited by 8,854
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  6. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  7. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  10. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  11. 2019Contrastive Representation Distillationcited by 734
  12. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
27 more
  1. 2017Deep Mutual Learningcited by 621
  2. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
  3. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 386
  4. 2018Graph Convolutional Networks for Text Classificationcited by 345
  5. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
  6. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 254
  7. 2021Does Knowledge Distillation Really Work?cited by 187
  8. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 169
  9. 2018Open Domain Question Answering Using Early Fusion of Knowledge Bases and Textcited by 167
  10. 2018Graph Convolution over Pruned Dependency Trees Improves Relation Extractioncited by 146
  11. 2019Attention Guided Graph Convolutional Networks for Relation Extractioncited by 82
  12. 2019Relational Graph Attention Networkscited by 81
  13. 2021BertGCN: Transductive Text Classification by Combining GCN and BERTcited by 74
  14. 2019Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphscited by 71
  15. 2020MixKD: Towards Efficient Distillation of Large-scale Language Modelscited by 65
  16. 2020Contrastive Distillation on Intermediate Representations for Language Model Compressioncited by 57
  17. 2020Do Syntax Trees Help Pre-trained Transformers Extract Information?cited by 47
  18. 2024Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semanticscited by 43
  19. 2020LRC-BERT: Latent-representation Contrastive Knowledge Distillation for Natural Language Understandingcited by 27
  20. 2023FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extractioncited by 20
  21. 2022Knowledge Graph Question Answering Datasets and Their Generalizability: Are They Enough for Future Research?cited by 13
  22. 2023Probing LLMs for Joint Encoding of Linguistic Categoriescited by 9
  23. 2017Symbolic, Distributed and Distributional Representations for Natural Language Processing in the Era of Deep Learning: a Surveycited by 8
  24. 2023A Graphical Approach to Document Layout Analysiscited by 8
  25. 2025Beyond Seen Data: Improving KBQA Generalization Through Schema-Guided Logical Form Generationcited by 8
  26. 2022Distilling Multi-Scale Knowledge for Event Temporal Relation Extractioncited by 6
  27. 2023Linguistic representations for fewer-shot relation extraction across domainscited by 4

Built on by

Not cited yet.