{"total":177,"rows":[{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268,"area":null},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133,"area":null},{"id":"2105.00304","title":"SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects","created":"2021-05-01","cited_by_count":104,"area":null},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102,"area":null},{"id":"2108.10573","title":"The staircase property: How hierarchical structure can guide deep learning","created":"2021-08-24","cited_by_count":62,"area":null},{"id":"2105.14835","title":"Towards Lower Bounds on the Depth of ReLU Neural Networks","created":"2021-05-31","cited_by_count":56,"area":null},{"id":"2412.02975","title":"Theoretical limitations of multi-layer Transformer","created":"2024-12-04","cited_by_count":45,"area":null},{"id":"2110.03187","title":"On the Optimal Memorization Power of ReLU Neural Networks","created":"2021-10-07","cited_by_count":40,"area":null},{"id":"2309.13414","title":"State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory","created":"2023-09-23","cited_by_count":39,"area":null},{"id":"2104.08135","title":"Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums","created":"2021-04-16","cited_by_count":37,"area":null},{"id":"2302.12553","title":"Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes","created":"2023-02-24","cited_by_count":36,"area":null},{"id":"2202.07626","title":"Random Feature Amplification: Feature Learning and Generalization in Neural Networks","created":"2022-02-15","cited_by_count":35,"area":null},{"id":"2102.02336","title":"On the Approximation Power of Two-Layer Networks of Random ReLUs","created":"2021-02-03","cited_by_count":29,"area":null},{"id":"2210.07236","title":"Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions","created":"2022-10-13","cited_by_count":26,"area":null},{"id":"2305.12972","title":"VanillaNet: the Power of Minimalism in Deep Learning","created":"2023-05-22","cited_by_count":23,"area":null},{"id":"2102.01670","title":"Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network Optimization","created":"2021-02-02","cited_by_count":22,"area":null},{"id":"2206.08615","title":"On the Number of Regions of Piecewise Linear Neural Networks","created":"2022-06-17","cited_by_count":21,"area":null},{"id":"2102.00434","title":"The Connection Between Approximation, Depth Separation and Learnability in Neural Networks","created":"2021-01-31","cited_by_count":19,"area":null},{"id":"2305.06986","title":"Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks","created":"2023-05-11","cited_by_count":19,"area":null},{"id":"2102.01621","title":"Depth separation beyond radial functions","created":"2021-02-02","cited_by_count":18,"area":null},{"id":"2309.10402","title":"Minimum width for universal approximation using ReLU networks on compact domain","created":"2023-09-19","cited_by_count":18,"area":null},{"id":"2502.06283","title":"On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth","created":"2025-02-10","cited_by_count":18,"area":null},{"id":"2102.10492","title":"Deep ReLU Networks Preserve Expected Length","created":"2021-02-21","cited_by_count":17,"area":null},{"id":"2112.02393","title":"Optimization-Based Separations for Neural Networks","created":"2021-12-04","cited_by_count":17,"area":null},{"id":"2112.05611","title":"Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks","created":"2021-12-10","cited_by_count":17,"area":null},{"id":"2209.08473","title":"Bootstrap Generalization Ability from Loss Landscape Perspective","created":"2022-09-18","cited_by_count":16,"area":null},{"id":"2107.13191","title":"Neural Network Approximation of Refinable Functions","created":"2021-07-28","cited_by_count":15,"area":null},{"id":"2110.05445","title":"Data-driven approaches for predicting spread of infectious diseases through DINNs: Disease Informed Neural Networks","created":"2021-10-11","cited_by_count":15,"area":null},{"id":"2204.01368","title":"Training Fully Connected Neural Networks is $\\exists\\mathbb{R}$-Complete","created":"2022-04-04","cited_by_count":15,"area":null},{"id":"2507.12549","title":"The Serial Scaling Hypothesis","created":"2025-07-16","cited_by_count":13,"area":null},{"id":"2102.06635","title":"ReLU Neural Networks of Polynomial Size for Exact Maximum Flow Computation","created":"2021-02-12","cited_by_count":12,"area":null},{"id":"2109.11354","title":"Arbitrary-Depth Universal Approximation Theorems for Operator Neural Networks","created":"2021-09-23","cited_by_count":12,"area":null},{"id":"2111.08117","title":"Neural networks with linear threshold activations: structure and algorithms","created":"2021-11-15","cited_by_count":12,"area":null},{"id":"2112.09747","title":"A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation","created":"2021-12-17","cited_by_count":12,"area":null},{"id":"2307.09212","title":"How Many Neurons Does it Take to Approximate the Maximum?","created":"2023-07-18","cited_by_count":12,"area":null},{"id":"2505.14338","title":"Better Neural Network Expressivity: Subdividing the Simplex","created":"2025-05-20","cited_by_count":12,"area":null},{"id":"2402.15315","title":"On Minimal Depth in Neural Networks","created":"2024-02-23","cited_by_count":11,"area":null},{"id":"2502.09324","title":"Depth-Bounds for Neural Networks via the Braid Arrangement","created":"2025-02-13","cited_by_count":11,"area":null},{"id":"2505.06169","title":"On the Depth of Monotone ReLU Neural Networks and ICNNs","created":"2025-05-09","cited_by_count":11,"area":null},{"id":"2202.03841","title":"Width is Less Important than Depth in ReLU Neural Networks","created":"2022-02-08","cited_by_count":10,"area":null},{"id":"2206.01266","title":"Exponential Separations in Symmetric Neural Networks","created":"2022-06-02","cited_by_count":10,"area":null},{"id":"2207.05275","title":"Size and depth of monotone neural networks: interpolation and approximation","created":"2022-07-12","cited_by_count":10,"area":null},{"id":"2209.10091","title":"Variational Inference for Infinitely Deep Neural Networks","created":"2022-09-21","cited_by_count":10,"area":null},{"id":"2211.08404","title":"Non-Linear Coordination Graphs","created":"2022-10-26","cited_by_count":10,"area":null},{"id":"2310.11130","title":"Topological Expressivity of ReLU Neural Networks","created":"2023-10-17","cited_by_count":10,"area":null},{"id":"2304.01063","title":"Depth Separation with Multilayer Mean-Field Networks","created":"2023-04-03","cited_by_count":9,"area":null},{"id":"2312.16483","title":"Expressivity and Approximation Properties of Deep Neural Networks with ReLU$^k$ Activation","created":"2023-12-27","cited_by_count":9,"area":null},{"id":"2401.09902","title":"Interplay between depth and width for interpolation in neural ODEs","created":"2024-01-18","cited_by_count":9,"area":null},{"id":"2410.04907","title":"Decomposition Polyhedra of Piecewise Linear Functions","created":"2024-10-07","cited_by_count":9,"area":null},{"id":"2411.03006","title":"Neural Networks and (Virtual) Extended Formulations","created":"2024-11-05","cited_by_count":9,"area":null},{"id":"2102.06358","title":"Min-Max-Plus Neural Networks","created":"2021-02-12","cited_by_count":8,"area":null},{"id":"2107.06304","title":"Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks","created":"2021-07-13","cited_by_count":8,"area":null},{"id":"2111.03135","title":"Scaffolding Sets","created":"2021-11-04","cited_by_count":8,"area":null},{"id":"2210.12001","title":"When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work","created":"2022-10-21","cited_by_count":8,"area":null},{"id":"2102.00314","title":"Size and Depth Separation in Approximating Benign Functions with Neural Networks","created":"2021-01-30","cited_by_count":7,"area":null},{"id":"2106.14749","title":"A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning","created":"2021-06-28","cited_by_count":7,"area":null},{"id":"2205.13283","title":"Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks","created":"2022-05-26","cited_by_count":7,"area":null},{"id":"2108.06711","title":"Towards Understanding Theoretical Advantages of Complex-Reaction Networks","created":"2021-08-15","cited_by_count":6,"area":null},{"id":"2601.19756","title":"Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining","created":"2026-01-27","cited_by_count":6,"area":null},{"id":"2605.03601","title":"Most ReLU Networks Admit Identifiable Parameters","created":"2026-05-05","cited_by_count":6,"area":null},{"id":"2102.05347","title":"From Sampling to Optimization on Discrete Domains with Applications to Determinant Maximization","created":"2021-02-10","cited_by_count":5,"area":null},{"id":"2107.09532","title":"Estimation of a regression function on a manifold by fully connected deep neural networks","created":"2021-07-20","cited_by_count":5,"area":null},{"id":"2107.12466","title":"High-Dimensional Distribution Generation Through Deep Neural Networks","created":"2021-07-26","cited_by_count":5,"area":null},{"id":"2204.01707","title":"Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection","created":"2022-04-02","cited_by_count":5,"area":null},{"id":"2204.03931","title":"HINNPerf: Hierarchical Interaction Neural Network for Performance Prediction of Configurable Systems","created":"2022-04-08","cited_by_count":5,"area":null},{"id":"2206.05317","title":"Intrinsic dimensionality and generalization properties of the $\\mathcal{R}$-norm inductive bias","created":"2022-06-10","cited_by_count":5,"area":null},{"id":"2307.01177","title":"Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space","created":"2023-07-03","cited_by_count":5,"area":null},{"id":"2101.04243","title":"A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks","created":"2021-01-12","cited_by_count":4,"area":null},{"id":"2104.05021","title":"CovNet: Covariance Networks for Functional Data on Multidimensional Domains","created":"2021-04-11","cited_by_count":4,"area":null},{"id":"2206.05904","title":"Theoretical guarantees for the advantage of GNNs over NNs in generalizing bandlimited functions on Euclidean cubes","created":"2022-06-13","cited_by_count":4,"area":null},{"id":"2209.01173","title":"Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality","created":"2022-09-02","cited_by_count":4,"area":null},{"id":"2312.13807","title":"Cluster-based classification with neural ODEs via control","created":"2023-12-21","cited_by_count":4,"area":null},{"id":"2402.07248","title":"Depth Separations in Neural Networks: Separating the Dimension from the Accuracy","created":"2024-02-11","cited_by_count":4,"area":null},{"id":"2403.09728","title":"Simulating Weighted Automata over Sequences and Trees with Transformers","created":"2024-03-12","cited_by_count":4,"area":null},{"id":"2504.00820","title":"Deep Generative Models: Complexity, Dimensionality, and Approximation","created":"2025-04-01","cited_by_count":4,"area":null},{"id":"2111.06027","title":"Theoretical Exploration of Flexible Transmitter Model","created":"2021-11-11","cited_by_count":3,"area":null},{"id":"2112.09741","title":"Envisioning Future Deep Learning Theories: Some Basic Concepts and Characteristics","created":"2021-12-17","cited_by_count":3,"area":null},{"id":"2201.12795","title":"Training Thinner and Deeper Neural Networks: Jumpstart Regularization","created":"2022-01-30","cited_by_count":3,"area":null},{"id":"2204.07261","title":"Convergence and Implicit Regularization Properties of Gradient Descent for Deep Residual Networks","created":"2022-04-14","cited_by_count":3,"area":null},{"id":"2205.10760","title":"CNNs Avoid Curse of Dimensionality by Learning on Patches","created":"2022-05-22","cited_by_count":3,"area":null},{"id":"2206.09182","title":"Coin Flipping Neural Networks","created":"2022-06-18","cited_by_count":3,"area":null},{"id":"2207.07612","title":"Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution","created":"2022-07-15","cited_by_count":3,"area":null},{"id":"2210.05212","title":"On Scrambling Phenomena for Randomly Initialized Recurrent Networks","created":"2022-10-11","cited_by_count":3,"area":null},{"id":"2210.16286","title":"A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks","created":"2022-10-28","cited_by_count":3,"area":null},{"id":"2302.00834","title":"Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Data","created":"2023-02-02","cited_by_count":3,"area":null},{"id":"2302.09712","title":"Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization","created":"2023-02-20","cited_by_count":3,"area":null},{"id":"2310.02250","title":"Why should autoencoders work?","created":"2023-10-03","cited_by_count":3,"area":null},{"id":"2310.18725","title":"The Evolution of the Interplay Between Input Distributions and Linear Regions in Networks","created":"2023-10-28","cited_by_count":3,"area":null},{"id":"2401.06122","title":"Manipulating Feature Visualizations with Gradient Slingshots","created":"2024-01-11","cited_by_count":3,"area":null},{"id":"2402.08808","title":"Depth Separation in Norm-Bounded Infinite-Width Neural Networks","created":"2024-02-13","cited_by_count":3,"area":null},{"id":"2405.09541","title":"Spectral complexity of deep neural networks","created":"2024-05-15","cited_by_count":3,"area":null},{"id":"2406.11463","title":"Just How Flexible are Neural Networks in Practice?","created":"2024-06-17","cited_by_count":3,"area":null},{"id":"2410.05465","title":"On the Expressive Power of Tree-Structured Probabilistic Circuits","created":"2024-10-07","cited_by_count":3,"area":null},{"id":"2510.11942","title":"On efficiently computable functions, deep networks and sparse compositionality","created":"2025-10-13","cited_by_count":3,"area":null},{"id":"2601.01417","title":"A Depth Hierarchy for Computing the Maximum in ReLU Networks via Extremal Graph Theory","created":"2026-01-04","cited_by_count":3,"area":null},{"id":"2105.07228","title":"Analysis of Structured Deep Kernel Networks","created":"2021-05-15","cited_by_count":2,"area":null},{"id":"2108.10825","title":"Adaptive Group Lasso Neural Network Models for Functions of Few Variables and Time-Dependent Data","created":"2021-08-24","cited_by_count":2,"area":null},{"id":"2109.01461","title":"Dive into Layers: Neural Network Capacity Bounding using Algebraic Geometry","created":"2021-09-03","cited_by_count":2,"area":null},{"id":"2110.10295","title":"Expressivity of Neural Networks via Chaotic Itineraries beyond Sharkovsky's Theorem","created":"2021-10-19","cited_by_count":2,"area":null}],"next_cursor":"100"}