# On the Dataless Training of Neural Networks

arXiv: https://arxiv.org/abs/2510.25962 · HTML: https://mlsanity.com/paper/2510.25962/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 42 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [A Review of Generalized Zero-Shot Learning Methods](https://mlsanity.com/paper/2011.08641/) (2020)
- [Zero-Shot Self-Supervised Learning for MRI Reconstruction](https://mlsanity.com/paper/2102.07737/) (2021)
- [Three Decades of Activations: A Comprehensive Survey of 400 Activation Functions for Neural Networks](https://mlsanity.com/paper/2402.09092/) (2024)
- [Controlling Continuous Relaxation for Combinatorial Optimization](https://mlsanity.com/paper/2309.16965/) (2023)
- [Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction](https://mlsanity.com/paper/2402.04097/) (2024)
- [Reheated Gradient-based Discrete Sampling for Combinatorial Optimization](https://mlsanity.com/paper/2503.04047/) (2025)
- [Accelerating Local Search for the Maximum Independent Set Problem](https://mlsanity.com/paper/1602.01659/) (2016)
- [Direct Amortized Likelihood Ratio Estimation](https://mlsanity.com/paper/2311.10571/) (2023)
- [Understanding Untrained Deep Models for Inverse Problems: Algorithms and Theory](https://mlsanity.com/paper/2502.18612/) (2025)
- [torchmSAT: A GPU-Accelerated Approximation To The Maximum Satisfiability Problem](https://mlsanity.com/paper/2402.03640/) (2024)
- [UGoDIT: Unsupervised Group Deep Image Prior Via Transferable Weights](https://mlsanity.com/paper/2505.11720/) (2025)

## Abstract (arXiv)

This paper surveys studies on the use of neural networks for optimization in the training-data-free setting. Specifically, we examine the dataless application of neural network architectures in optimization by re-parameterizing problems using fully connected (or MLP), convolutional, graph, and quadratic neural networks. Although MLPs have been used to solve linear programs a few decades ago, this approach has recently gained increasing attention due to its promising results across diverse applications, including those based on combinatorial optimization, inverse problems, and partial differential equations. The motivation for this setting stems from two key (possibly over-lapping) factors: (i) data-driven learning approaches are still underdeveloped and have yet to demonstrate strong results, as seen in combinatorial optimization, and (ii) the availability of training data is inherently limited, such as in medical image reconstruction and other scientific applications. In this paper, we define the dataless setting and categorize it into two variants based on how a problem instance -- defined by a single datum -- is encoded onto the neural network: (i) architecture-agnostic methods and (ii) architecture-specific methods. Additionally, we discuss similarities and clarify distinctions between the dataless neural network (dNN) settings and related concepts such as zero-shot learning, one-shot learning, lifting in optimization, and over-parameterization.