# VisAdj: Learning Adjacency Matrices from Node-Link Images

arXiv: https://arxiv.org/abs/2608.21825 · HTML: https://mlsanity.com/paper/2608.21825/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 50 references that are in the map.
- First posted on arXiv 22 Aug 2026.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [SpaceNet: A Remote Sensing Dataset and Challenge Series](https://mlsanity.com/paper/1807.01232/) (2018)
- [Sparse Sequence-to-Sequence Models](https://mlsanity.com/paper/1905.05702/) (2019)
- [Adaptively Sparse Transformers](https://mlsanity.com/paper/1909.00015/) (2019)
- [Learning non-maximum suppression](https://mlsanity.com/paper/1705.02950/) (2017)
- [RoadTracer: Automatic Extraction of Road Networks from Aerial Images](https://mlsanity.com/paper/1802.03680/) (2018)
- [Relationformer: A Unified Framework for Image-to-Graph Generation](https://mlsanity.com/paper/2203.10202/) (2022)
- [Segment Anything Model for Road Network Graph Extraction](https://mlsanity.com/paper/2403.16051/) (2024)
- [RNGDet++: Road Network Graph Detection by Transformer with Instance Segmentation and Multi-scale Features Enhancement](https://mlsanity.com/paper/2209.10150/) (2022)
- [Image-Conditioned Graph Generation for Road Network Extraction](https://mlsanity.com/paper/1910.14388/) (2019)
- [Towards Satellite Image Road Graph Extraction: A Global-Scale Dataset and A Novel Method](https://mlsanity.com/paper/2411.16733/) (2024)
- [Node, Node-Link, and Node-Link-Group Diagrams: An Evaluation](https://mlsanity.com/paper/1404.1911/) (2014)
- [Cross-domain and Cross-dimension Learning for Image-to-Graph Transformers](https://mlsanity.com/paper/2403.06601/) (2024)
- [VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection](https://mlsanity.com/paper/2602.13880/) (2026)

## Abstract (arXiv)

Learning adjacency matrices from node-link images is a fundamental problem for recovering structured graph information from visual observations. Existing methods typically rely on fixed KNN-based heuristics for candidate edge selection and fail to capture dependencies among edges. To overcome these limitations, we propose VisAdj, a new framework for topology-aware adjacency prediction. VisAdj introduces an attention-sparse neighbor sampler to adaptively select a high-recall set of candidate node pairs and performs joint edge inference using a line-graph transformer that treats candidate edges as tokens and explicitly models dependencies among incident edges. Extensive experiments on synthetic graphs, road networks, and vessel images demonstrate that VisAdj consistently outperforms existing baselines by clear margins.