# A Multi-Resolution Benchmark Framework for Spatial Reasoning Assessment in Neural Networks

arXiv: https://arxiv.org/abs/2508.12741 · HTML: https://mlsanity.com/paper/2508.12741/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 18 Aug 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 9 of 9)

- [clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation](https://mlsanity.com/paper/2003.07311/) (2020)
- [Visual Spatial Reasoning](https://mlsanity.com/paper/2205.00363/) (2022)
- [nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation](https://mlsanity.com/paper/2404.09556/) (2024)
- [A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology](https://mlsanity.com/paper/1910.01877/) (2019)
- [Understanding metric-related pitfalls in image analysis validation](https://mlsanity.com/paper/2302.01790/) (2023)
- [Deep Vessel Segmentation By Learning Graphical Connectivity](https://mlsanity.com/paper/1806.02279/) (2018)
- [A persistent homology-based topological loss for CNN-based multi-class segmentation of CMR](https://mlsanity.com/paper/2107.12689/) (2021)
- [VoxLogicA: a Spatial Model Checker for Declarative Image Analysis (Extended Version)](https://mlsanity.com/paper/1811.05677/) (2018)
- [Universal Topology Refinement for Medical Image Segmentation with Polynomial Feature Synthesis](https://mlsanity.com/paper/2409.09796/) (2024)

## Abstract (arXiv)

This paper presents preliminary results in the definition of a comprehensive benchmark framework designed to systematically evaluate spatial reasoning capabilities in neural networks, with a particular focus on morphological properties such as connectivity and distance relationships. The framework is currently being used to study the capabilities of nnU-Net, exploiting the spatial model checker VoxLogicA to generate two distinct categories of synthetic datasets: maze connectivity problems for topological analysis and spatial distance computation tasks for geometric understanding. Each category is evaluated across multiple resolutions to assess scalability and generalization properties. The automated pipeline encompasses a complete machine learning workflow including: synthetic dataset generation, standardized training with cross-validation, inference execution, and comprehensive evaluation using Dice coefficient and IoU (Intersection over Union) metrics. Preliminary experimental results demonstrate significant challenges in neural network spatial reasoning capabilities, revealing systematic failures in basic geometric and topological understanding tasks. The framework provides a reproducible experimental protocol, enabling researchers to identify specific limitations. Such limitations could be addressed through hybrid approaches combining neural networks with symbolic reasoning methods for improved spatial understanding in clinical applications, establishing a foundation for ongoing research into neural network spatial reasoning limitations and potential solutions.