# Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answering

arXiv: https://arxiv.org/abs/2411.06866 · HTML: https://mlsanity.com/paper/2411.06866/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Builds on (most cited first, 13 of 13)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [GreaseLM: Graph REASoning Enhanced Language Models for Question Answering](https://mlsanity.com/paper/2201.08860/) (2022)
- [Deep Bidirectional Language-Knowledge Graph Pretraining](https://mlsanity.com/paper/2210.09338/) (2022)
- [Efficient Tuning and Inference for Large Language Models on Textual Graphs](https://mlsanity.com/paper/2401.15569/) (2024)
- [ConGraT: Self-Supervised Contrastive Pretraining for Joint Graph and Text Embeddings](https://mlsanity.com/paper/2305.14321/) (2023)
- [Graph-based Modeling of Online Communities for Fake News Detection](https://mlsanity.com/paper/2008.06274/) (2020)
- [GNN is a Counter? Revisiting GNN for Question Answering](https://mlsanity.com/paper/2110.03192/) (2021)
- [Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques](https://mlsanity.com/paper/2402.05952/) (2024)
- [Dynamic Relevance Graph Network for Knowledge-Aware Question Answering](https://mlsanity.com/paper/2209.09947/) (2022)

## Built on by (most cited first, 2 of 2)

- [UORA: Uniform Orthogonal Reinitialization Adaptation in Parameter-Efficient Fine-Tuning of Large Models](https://mlsanity.com/paper/2505.20154/) (2025)
- [Learning to Recommend Multi-Agent Subgraphs from Calling Trees](https://mlsanity.com/paper/2601.22209/) (2026)

## Abstract (arXiv)

Commonsense question answering is a crucial task that requires machines to employ reasoning according to commonsense. Previous studies predominantly employ an extracting-and-modeling paradigm to harness the information in KG, which first extracts relevant subgraphs based on pre-defined rules and then proceeds to design various strategies aiming to improve the representations and fusion of the extracted structural knowledge. Despite their effectiveness, there are still two challenges. On one hand, subgraphs extracted by rule-based methods may have the potential to overlook critical nodes and result in uncontrollable subgraph size. On the other hand, the misalignment between graph and text modalities undermines the effectiveness of knowledge fusion, ultimately impacting the task performance. To deal with the problems above, we propose a novel framework: Subgraph REtrieval Enhanced by GraPh-Text Alignment, named SEPTA. Firstly, we transform the knowledge graph into a database of subgraph vectors and propose a BFS-style subgraph sampling strategy to avoid information loss, leveraging the analogy between BFS and the message-passing mechanism. In addition, we propose a bidirectional contrastive learning approach for graph-text alignment, which effectively enhances both subgraph retrieval and knowledge fusion. Finally, all the retrieved information is combined for reasoning in the prediction module. Extensive experiments on five datasets demonstrate the effectiveness and robustness of our framework.