# On convex decision regions in deep network representations

arXiv: https://arxiv.org/abs/2305.17154 · HTML: https://mlsanity.com/paper/2305.17154/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 72 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [Complexity of Linear Regions in Deep Networks](https://mlsanity.com/paper/1901.09021/) (2019)
- [What do Neural Machine Translation Models Learn about Morphology?](https://mlsanity.com/paper/1704.03471/) (2017)
- [The geometry of hidden representations of large transformer models](https://mlsanity.com/paper/2302.00294/) (2023)
- [Learning Independent Causal Mechanisms](https://mlsanity.com/paper/1712.00961/) (2017)
- [Self-supervised Learning for Human Activity Recognition Using 700,000 Person-days of Wearable Data](https://mlsanity.com/paper/2206.02909/) (2022)
- [Neural population geometry: An approach for understanding biological and artificial neural networks](https://mlsanity.com/paper/2104.07059/) (2021)
- [SqueezeBERT: What can computer vision teach NLP about efficient neural networks?](https://mlsanity.com/paper/2006.11316/) (2020)
- [Emergence of Separable Manifolds in Deep Language Representations](https://mlsanity.com/paper/2006.01095/) (2020)
- [Geodesics of learned representations](https://mlsanity.com/paper/1511.06394/) (2015)
- [Approximating geodesics via random points](https://mlsanity.com/paper/1711.06952/) (2017)
- [Deep ReLU Networks Have Surprisingly Simple Polytopes](https://mlsanity.com/paper/2305.09145/) (2023)
- [Convexity in complex networks](https://mlsanity.com/paper/1608.03402/) (2016)
- [Grounding Psychological Shape Space in Convolutional Neural Networks](https://mlsanity.com/paper/2111.08409/) (2021)

## Built on by (most cited first, 7 of 7)

- [Do Sparse Autoencoders Capture Concept Manifolds?](https://mlsanity.com/paper/2604.28119/) (2026)
- [Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds](https://mlsanity.com/paper/2606.25234/) (2026)
- [Large Vision Models Can Solve Mental Rotation Problems](https://mlsanity.com/paper/2509.15271/) (2025)
- [Connecting Concept Convexity and Human-Machine Alignment in Deep Neural Networks](https://mlsanity.com/paper/2409.06362/) (2024)
- [Parametrizing Convex Sets Using Sublinear Neural Networks](https://mlsanity.com/paper/2605.03520/) (2026)
- [A Navigable Manifold of Hypothesized Consciousness-Spectrum States in Language Model Representations](https://mlsanity.com/paper/2606.09894/) (2026)
- [How Molecular Generative Models Organize Molecular Identity](https://mlsanity.com/paper/2608.06956/) (2026)

## Abstract (arXiv)

Current work on human-machine alignment aims at understanding machine-learned latent spaces and their correspondence to human representations. G"ardenfors' conceptual spaces is a prominent framework for understanding human representations. Convexity of object regions in conceptual spaces is argued to promote generalizability, few-shot learning, and interpersonal alignment. Based on these insights, we investigate the notion of convexity of concept regions in machine-learned latent spaces. We develop a set of tools for measuring convexity in sampled data and evaluate emergent convexity in layered representations of state-of-the-art deep networks. We show that convexity is robust to basic re-parametrization and, hence, meaningful as a quality of machine-learned latent spaces. We find that approximate convexity is pervasive in neural representations in multiple application domains, including models of images, audio, human activity, text, and medical images. Generally, we observe that fine-tuning increases the convexity of label regions. We find evidence that pretraining convexity of class label regions predicts subsequent fine-tuning performance.