# DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning

arXiv: https://arxiv.org/abs/2610.16822 · HTML: https://mlsanity.com/paper/2610.16822/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware](https://mlsanity.com/paper/1812.00332/) (2018)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [NAS-Bench-101: Towards Reproducible Neural Architecture Search](https://mlsanity.com/paper/1902.09635/) (2019)
- [Representing Long-Range Context for Graph Neural Networks with Global Attention](https://mlsanity.com/paper/2201.08821/) (2022)
- [Neural Architecture Optimization](https://mlsanity.com/paper/1808.07233/) (2018)
- [Neural Predictor for Neural Architecture Search](https://mlsanity.com/paper/1912.00848/) (2019)
- [BRP-NAS: Prediction-based NAS using GCNs](https://mlsanity.com/paper/2007.08668/) (2020)
- [MagNet: A Neural Network for Directed Graphs](https://mlsanity.com/paper/2102.11391/) (2021)
- [A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAS](https://mlsanity.com/paper/2004.01899/) (2020)
- [A Large-Scale Database for Graph Representation Learning](https://mlsanity.com/paper/2011.07682/) (2020)
- [Semi-Supervised Neural Architecture Search](https://mlsanity.com/paper/2002.10389/) (2020)
- [Path-Level Network Transformation for Efficient Architecture Search](https://mlsanity.com/paper/1806.02639/) (2018)
- [ReNAS:Relativistic Evaluation of Neural Architecture Search](https://mlsanity.com/paper/1910.01523/) (2019)
- [Peephole: Predicting Network Performance Before Training](https://mlsanity.com/paper/1712.03351/) (2017)
- [NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction](https://mlsanity.com/paper/2211.08024/) (2022)
- [Killing Two Birds with One Stone: Unifying Retrieval and Ranking with a Single Generative Recommendation Model](https://mlsanity.com/paper/2504.16454/) (2025)
- [FlowerFormer: Empowering Neural Architecture Encoding using a Flow-aware Graph Transformer](https://mlsanity.com/paper/2403.12821/) (2024)
- [NAR-Former V2: Rethinking Transformer for Universal Neural Network Representation Learning](https://mlsanity.com/paper/2306.10792/) (2023)
- [NN-Former: Rethinking Graph Structure in Neural Architecture Representation](https://mlsanity.com/paper/2507.00880/) (2025)
- [Tools as Continuous Flow for Evolving Agentic Reasoning](https://mlsanity.com/paper/2605.07339/) (2026)
- [LiBrA-Net: Lie-Algebraic Bilateral Affine Fields for Real-Time 4K Video Dehazing](https://mlsanity.com/paper/2605.11508/) (2026)

## Abstract (arXiv)

Architecture attribute predictors reduce the cost of neural architecture search by estimating accuracy or latency from a small set of evaluated candidates. The key difficulty is that neural architectures are operation-labeled DAGs: edge direction defines computational flow, but existing predictors often treat direction as a local structural cue. We propose DTA-GT, a Direction- and Topology-Aware Graph Transformer based on encoder-wide directional consistency, which preserves directed computational semantics across node initialization, global spectral encoding, pairwise interaction, and feature update. DTA-GT realizes this principle with direction- and topology-aware node initialization, Magnetic Laplacian spectral encoding, direction-dual structural attention, and a Direction-Sensitive MoE-FFN. Across accuracy and latency prediction on NAS-Bench-101 and NAS-Bench-201, DTA-GT consistently outperforms representative sequence-, GNN-, Transformer-, and hybrid predictors under limited-budget settings. Ablations and Magnetic spectral controls show that the improvements come from coordinated direction-aware stages and edge-orientation-sensitive spectral information, while mechanistic diagnostics confirm that the modules exhibit the intended directional behavior. These results support encoder-wide directional consistency as an effective design principle for architecture DAG representation.