{"total":236,"rows":[{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268,"area":null},{"id":"2103.03212","title":"Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks","created":"2021-03-04","cited_by_count":221,"area":null},{"id":"2203.05962","title":"Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice","created":"2022-03-09","cited_by_count":136,"area":null},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133,"area":null},{"id":"2105.00304","title":"SpookyNet: Learning Force Fields with Electronic Degrees of Freedom and Nonlocal Effects","created":"2021-05-01","cited_by_count":104,"area":null},{"id":"2107.13163","title":"Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers","created":"2021-07-28","cited_by_count":102,"area":null},{"id":"2402.09268","title":"Transformers, parallel computation, and logarithmic depth","created":"2024-02-14","cited_by_count":79,"area":null},{"id":"2112.08348","title":"Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts","created":"2021-12-15","cited_by_count":64,"area":null},{"id":"2108.10573","title":"The staircase property: How hierarchical structure can guide deep learning","created":"2021-08-24","cited_by_count":62,"area":null},{"id":"2105.14835","title":"Towards Lower Bounds on the Depth of ReLU Neural Networks","created":"2021-05-31","cited_by_count":56,"area":null},{"id":"2211.14400","title":"Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces","created":"2022-11-25","cited_by_count":51,"area":null},{"id":"2412.02975","title":"Theoretical limitations of multi-layer Transformer","created":"2024-12-04","cited_by_count":45,"area":null},{"id":"2110.03187","title":"On the Optimal Memorization Power of ReLU Neural Networks","created":"2021-10-07","cited_by_count":40,"area":null},{"id":"2402.04161","title":"Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains","created":"2024-02-06","cited_by_count":40,"area":null},{"id":"2104.08135","title":"Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums","created":"2021-04-16","cited_by_count":37,"area":null},{"id":"2302.12553","title":"Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopes","created":"2023-02-24","cited_by_count":36,"area":null},{"id":"2512.01809","title":"Much Ado About Noising: Dispelling the Myths of Generative Robotic Control","created":"2025-12-01","cited_by_count":36,"area":null},{"id":"2202.07626","title":"Random Feature Amplification: Feature Learning and Generalization in Neural Networks","created":"2022-02-15","cited_by_count":35,"area":null},{"id":"2205.01510","title":"ExSpliNet: An interpretable and expressive spline-based neural network","created":"2022-05-03","cited_by_count":33,"area":null},{"id":"2104.02746","title":"Proof of the Theory-to-Practice Gap in Deep Learning via Sampling Complexity bounds for Neural Network Approximation Spaces","created":"2021-04-06","cited_by_count":30,"area":null},{"id":"2111.11146","title":"On the Existence of Universal Lottery Tickets","created":"2021-11-22","cited_by_count":30,"area":null},{"id":"2102.02336","title":"On the Approximation Power of Two-Layer Networks of Random ReLUs","created":"2021-02-03","cited_by_count":29,"area":null},{"id":"2309.16314","title":"A Primer on Bayesian Neural Networks: Review and Debates","created":"2023-09-28","cited_by_count":27,"area":null},{"id":"2106.00553","title":"SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit models","created":"2021-06-01","cited_by_count":26,"area":null},{"id":"2209.11395","title":"Achieve the Minimum Width of Neural Networks for Universal Approximation","created":"2022-09-23","cited_by_count":26,"area":null},{"id":"2210.07236","title":"Improved Bounds on Neural Complexity for Representing Piecewise Linear Functions","created":"2022-10-13","cited_by_count":26,"area":null},{"id":"2101.06154","title":"On the statistical complexity of quantum circuits","created":"2021-01-15","cited_by_count":25,"area":null},{"id":"2106.13361","title":"Multifidelity Modeling for Physics-Informed Neural Networks (PINNs)","created":"2021-06-25","cited_by_count":25,"area":null},{"id":"2203.10418","title":"How do noise tails impact on deep ReLU networks?","created":"2022-03-20","cited_by_count":25,"area":null},{"id":"2403.11871","title":"The Real Tropical Geometry of Neural Networks","created":"2024-03-18","cited_by_count":24,"area":null},{"id":"2407.15160","title":"When Can Transformers Count to n?","created":"2024-07-21","cited_by_count":24,"area":null},{"id":"2205.13863","title":"Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power","created":"2022-05-27","cited_by_count":23,"area":null},{"id":"2305.12972","title":"VanillaNet: the Power of Minimalism in Deep Learning","created":"2023-05-22","cited_by_count":23,"area":null},{"id":"2107.00896","title":"Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions","created":"2021-07-02","cited_by_count":22,"area":null},{"id":"2106.06529","title":"The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective","created":"2021-06-11","cited_by_count":21,"area":null},{"id":"2211.03232","title":"Exponentially Improving the Complexity of Simulating the Weisfeiler-Lehman Test with Graph Neural Networks","created":"2022-11-06","cited_by_count":21,"area":null},{"id":"2102.00434","title":"The Connection Between Approximation, Depth Separation and Learnability in Neural Networks","created":"2021-01-31","cited_by_count":19,"area":null},{"id":"2305.06986","title":"Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks","created":"2023-05-11","cited_by_count":19,"area":null},{"id":"2102.01621","title":"Depth separation beyond radial functions","created":"2021-02-02","cited_by_count":18,"area":null},{"id":"2309.10402","title":"Minimum width for universal approximation using ReLU networks on compact domain","created":"2023-09-19","cited_by_count":18,"area":null},{"id":"2502.06283","title":"On the Expressiveness of Rational ReLU Neural Networks With Bounded Depth","created":"2025-02-10","cited_by_count":18,"area":null},{"id":"2102.10492","title":"Deep ReLU Networks Preserve Expected Length","created":"2021-02-21","cited_by_count":17,"area":null},{"id":"2112.02393","title":"Optimization-Based Separations for Neural Networks","created":"2021-12-04","cited_by_count":17,"area":null},{"id":"2101.11256","title":"Partition of unity networks: deep hp-approximation","created":"2021-01-27","cited_by_count":16,"area":null},{"id":"2107.00379","title":"On the Expected Complexity of Maxout Networks","created":"2021-07-01","cited_by_count":15,"area":null},{"id":"2107.13191","title":"Neural Network Approximation of Refinable Functions","created":"2021-07-28","cited_by_count":15,"area":null},{"id":"2109.00190","title":"Approximation Properties of Deep ReLU CNNs","created":"2021-09-01","cited_by_count":15,"area":null},{"id":"2204.01368","title":"Training Fully Connected Neural Networks is $\\exists\\mathbb{R}$-Complete","created":"2022-04-04","cited_by_count":15,"area":null},{"id":"2303.00579","title":"Are More Layers Beneficial to Graph Transformers?","created":"2023-03-01","cited_by_count":15,"area":null},{"id":"2106.09309","title":"Layer Folding: Neural Network Depth Reduction using Activation Linearization","created":"2021-06-17","cited_by_count":14,"area":null},{"id":"2306.07212","title":"Polyhedral Complex Extraction from ReLU Networks using Edge Subdivision","created":"2023-06-12","cited_by_count":14,"area":null},{"id":"2408.04569","title":"Activation degree thresholds and expressiveness of polynomial neural networks","created":"2024-08-08","cited_by_count":14,"area":null},{"id":"2202.11749","title":"Are All Linear Regions Created Equal?","created":"2022-02-23","cited_by_count":13,"area":null},{"id":"2305.18460","title":"Minimum Width of Leaky-ReLU Neural Networks for Uniform Universal Approximation","created":"2023-05-29","cited_by_count":13,"area":null},{"id":"2410.08194","title":"Features are fate: a theory of transfer learning in high-dimensional regression","created":"2024-10-10","cited_by_count":13,"area":null},{"id":"2507.12549","title":"The Serial Scaling Hypothesis","created":"2025-07-16","cited_by_count":13,"area":null},{"id":"2102.06635","title":"ReLU Neural Networks of Polynomial Size for Exact Maximum Flow Computation","created":"2021-02-12","cited_by_count":12,"area":null},{"id":"2105.04046","title":"A likelihood approach to nonparametric estimation of a singular distribution using deep generative models","created":"2021-05-09","cited_by_count":12,"area":null},{"id":"2105.04156","title":"ReLU Deep Neural Networks from the Hierarchical Basis Perspective","created":"2021-05-10","cited_by_count":12,"area":null},{"id":"2106.03186","title":"Reverse Engineering the Neural Tangent Kernel","created":"2021-06-06","cited_by_count":12,"area":null},{"id":"2109.11354","title":"Arbitrary-Depth Universal Approximation Theorems for Operator Neural Networks","created":"2021-09-23","cited_by_count":12,"area":null},{"id":"2111.08117","title":"Neural networks with linear threshold activations: structure and algorithms","created":"2021-11-15","cited_by_count":12,"area":null},{"id":"2206.04360","title":"A general approximation lower bound in $L^p$ norm, with applications to feed-forward neural networks","created":"2022-06-09","cited_by_count":12,"area":null},{"id":"2211.14810","title":"A Kernel Perspective of Skip Connections in Convolutional Networks","created":"2022-11-27","cited_by_count":12,"area":null},{"id":"2307.09212","title":"How Many Neurons Does it Take to Approximate the Maximum?","created":"2023-07-18","cited_by_count":12,"area":null},{"id":"2505.14338","title":"Better Neural Network Expressivity: Subdividing the Simplex","created":"2025-05-20","cited_by_count":12,"area":null},{"id":"2105.09673","title":"An Exact Poly-Time Membership-Queries Algorithm for Extraction a three-Layer ReLU Network","created":"2021-05-20","cited_by_count":11,"area":null},{"id":"2205.14819","title":"Universality of Group Convolutional Neural Networks Based on Ridgelet Analysis on Groups","created":"2022-05-30","cited_by_count":11,"area":null},{"id":"2301.12309","title":"On the Lipschitz Constant of Deep Networks and Double Descent","created":"2023-01-28","cited_by_count":11,"area":null},{"id":"2402.15315","title":"On Minimal Depth in Neural Networks","created":"2024-02-23","cited_by_count":11,"area":null},{"id":"2502.09324","title":"Depth-Bounds for Neural Networks via the Braid Arrangement","created":"2025-02-13","cited_by_count":11,"area":null},{"id":"2505.06169","title":"On the Depth of Monotone ReLU Neural Networks and ICNNs","created":"2025-05-09","cited_by_count":11,"area":null},{"id":"2202.03841","title":"Width is Less Important than Depth in ReLU Neural Networks","created":"2022-02-08","cited_by_count":10,"area":null},{"id":"2206.01266","title":"Exponential Separations in Symmetric Neural Networks","created":"2022-06-02","cited_by_count":10,"area":null},{"id":"2310.11130","title":"Topological Expressivity of ReLU Neural Networks","created":"2023-10-17","cited_by_count":10,"area":null},{"id":"2212.05427","title":"Statistical guarantees for sparse deep learning","created":"2022-12-11","cited_by_count":9,"area":null},{"id":"2304.01063","title":"Depth Separation with Multilayer Mean-Field Networks","created":"2023-04-03","cited_by_count":9,"area":null},{"id":"2407.09505","title":"1-Lipschitz Neural Distance Fields","created":"2024-06-14","cited_by_count":9,"area":null},{"id":"2410.04907","title":"Decomposition Polyhedra of Piecewise Linear Functions","created":"2024-10-07","cited_by_count":9,"area":null},{"id":"2411.03006","title":"Neural Networks and (Virtual) Extended Formulations","created":"2024-11-05","cited_by_count":9,"area":null},{"id":"2412.18283","title":"On the Local Complexity of Linear Regions in Deep ReLU Networks","created":"2024-12-24","cited_by_count":9,"area":null},{"id":"2506.07958","title":"Neural Tangent Kernel Analysis to Probe Convergence in Physics-informed Neural Solvers: PIKANs vs. PINNs","created":"2025-06-09","cited_by_count":9,"area":null},{"id":"2107.06304","title":"Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks","created":"2021-07-13","cited_by_count":8,"area":null},{"id":"2210.12001","title":"When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work","created":"2022-10-21","cited_by_count":8,"area":null},{"id":"2304.04443","title":"Approximation of Nonlinear Functionals Using Deep ReLU Networks","created":"2023-04-10","cited_by_count":8,"area":null},{"id":"2102.00314","title":"Size and Depth Separation in Approximating Benign Functions with Neural Networks","created":"2021-01-30","cited_by_count":7,"area":null},{"id":"2106.14749","title":"A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning","created":"2021-06-28","cited_by_count":7,"area":null},{"id":"2205.07764","title":"On the inability of Gaussian process regression to optimally learn compositional functions","created":"2022-05-16","cited_by_count":7,"area":null},{"id":"2205.13283","title":"Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks","created":"2022-05-26","cited_by_count":7,"area":null},{"id":"2206.00241","title":"Asymptotic Properties for Bayesian Neural Network in Besov Space","created":"2022-06-01","cited_by_count":7,"area":null},{"id":"2310.01691","title":"Zero-Shot Continuous Prompt Transfer: Generalizing Task Semantics Across Language Models","created":"2023-10-02","cited_by_count":7,"area":null},{"id":"2401.02890","title":"Nonlinear functional regression by functional deep neural network with kernel embedding","created":"2024-01-05","cited_by_count":7,"area":null},{"id":"2501.09283","title":"Free-Knots Kolmogorov-Arnold Network: On the Analysis of Spline Knots and Advancing Stability","created":"2025-01-16","cited_by_count":6,"area":null},{"id":"2601.19756","title":"Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining","created":"2026-01-27","cited_by_count":6,"area":null},{"id":"2605.03601","title":"Most ReLU Networks Admit Identifiable Parameters","created":"2026-05-05","cited_by_count":6,"area":null},{"id":"2107.12466","title":"High-Dimensional Distribution Generation Through Deep Neural Networks","created":"2021-07-26","cited_by_count":5,"area":null},{"id":"2206.05317","title":"Intrinsic dimensionality and generalization properties of the $\\mathcal{R}$-norm inductive bias","created":"2022-06-10","cited_by_count":5,"area":null},{"id":"2307.01177","title":"Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space","created":"2023-07-03","cited_by_count":5,"area":null},{"id":"2407.16153","title":"On the Benefits of Rank in Attention Layers","created":"2024-07-23","cited_by_count":5,"area":null},{"id":"2602.10867","title":"Deep Learning of Compositional Targets with Hierarchical Spectral Methods","created":"2026-02-11","cited_by_count":5,"area":null}],"next_cursor":"100"}