{"total":118,"rows":[{"id":"2407.05000","title":"LoRA-GA: Low-Rank Adaptation with Gradient Approximation","created":"2024-07-06","cited_by_count":141,"area":null},{"id":"2112.09153","title":"An Empirical Investigation of the Role of Pre-training in Lifelong Learning","created":"2021-12-16","cited_by_count":134,"area":null},{"id":"2101.08692","title":"Characterizing signal propagation to close the performance gap in unnormalized ResNets","created":"2021-01-21","cited_by_count":88,"area":null},{"id":"2303.00980","title":"Learning to Grow Pretrained Models for Efficient Transformer Training","created":"2023-03-02","cited_by_count":83,"area":null},{"id":"2107.11472","title":"Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers","created":"2021-07-23","cited_by_count":57,"area":null},{"id":"2110.01765","title":"Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping","created":"2021-10-05","cited_by_count":52,"area":null},{"id":"2111.05193","title":"A Survey on Green Deep Learning","created":"2021-11-08","cited_by_count":52,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2102.08098","title":"GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training","created":"2021-02-16","cited_by_count":44,"area":null},{"id":"2103.14021","title":"Orthogonal Projection Loss","created":"2021-03-25","cited_by_count":44,"area":null},{"id":"2203.06020","title":"Enhancing Adversarial Training with Second-Order Statistics of Weights","created":"2022-03-11","cited_by_count":42,"area":null},{"id":"2311.18823","title":"Initializing Models with Larger Ones","created":"2023-11-30","cited_by_count":39,"area":null},{"id":"2111.13293","title":"KNAS: Green Neural Architecture Search","created":"2021-11-26","cited_by_count":33,"area":null},{"id":"2203.11316","title":"Random vector functional link network: recent developments, applications, and future directions","created":"2022-02-13","cited_by_count":33,"area":null},{"id":"2304.03589","title":"On Efficient Training of Large-Scale Deep Learning Models: A Literature Review","created":"2023-04-07","cited_by_count":26,"area":null},{"id":"2211.07844","title":"Characterizing the Spectrum of the NTK via a Power Series Expansion","created":"2022-11-15","cited_by_count":22,"area":null},{"id":"2410.07170","title":"Parameter Efficient Fine-tuning via Explained Variance Adaptation","created":"2024-10-09","cited_by_count":20,"area":null},{"id":"2103.13549","title":"An evidential classifier based on Dempster-Shafer theory and deep learning","created":"2021-03-25","cited_by_count":18,"area":null},{"id":"2301.05219","title":"Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning","created":"2023-01-12","cited_by_count":16,"area":null},{"id":"2304.14994","title":"A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks","created":"2023-04-28","cited_by_count":16,"area":null},{"id":"2312.05614","title":"Transformer as Linear Expansion of Learngene","created":"2023-12-09","cited_by_count":13,"area":null},{"id":"2312.12474","title":"Principled Weight Initialisation for Input-Convex Neural Networks","created":"2023-12-19","cited_by_count":12,"area":null},{"id":"2210.05956","title":"Towards Theoretically Inspired Neural Initialization Optimization","created":"2022-10-12","cited_by_count":11,"area":null},{"id":"2409.19315","title":"Analog In-Memory Computing Attention Mechanism for Fast and Energy-Efficient Large Language Models","created":"2024-09-28","cited_by_count":11,"area":null},{"id":"2509.03417","title":"Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study","created":"2025-09-03","cited_by_count":11,"area":null},{"id":"2310.07235","title":"Are GATs Out of Balance?","created":"2023-10-11","cited_by_count":10,"area":null},{"id":"2102.10499","title":"A Hierarchical Conditional Random Field-based Attention Mechanism Approach for Gastric Histopathology Image Classification","created":"2021-02-21","cited_by_count":9,"area":null},{"id":"2108.06530","title":"Neuron Campaign for Initialization Guided by Information Bottleneck Theory","created":"2021-08-14","cited_by_count":9,"area":null},{"id":"2109.12960","title":"Ridgeless Interpolation with Shallow ReLU Networks in $1D$ is Nearest Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz Functions","created":"2021-09-27","cited_by_count":9,"area":null},{"id":"2212.05427","title":"Statistical guarantees for sparse deep learning","created":"2022-12-11","cited_by_count":9,"area":null},{"id":"2404.16897","title":"Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models","created":"2024-04-25","cited_by_count":9,"area":null},{"id":"2501.04315","title":"RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation","created":"2025-01-08","cited_by_count":9,"area":null},{"id":"2209.14778","title":"Batch Normalization Explained","created":"2022-09-29","cited_by_count":8,"area":null},{"id":"2304.05758","title":"Best Practices for 2-Body Pose Forecasting","created":"2023-04-12","cited_by_count":8,"area":null},{"id":"2308.14929","title":"Maestro: Uncovering Low-Rank Structures via Trainable Decomposition","created":"2023-08-28","cited_by_count":8,"area":null},{"id":"2310.02012","title":"Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion","created":"2023-10-03","cited_by_count":8,"area":null},{"id":"2208.01134","title":"Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization","created":"2022-08-01","cited_by_count":7,"area":null},{"id":"2201.09209","title":"Weight Expansion: A New Perspective on Dropout and Generalization","created":"2022-01-23","cited_by_count":6,"area":null},{"id":"2101.02338","title":"Max-Affine Spline Insights Into Deep Network Pruning","created":"2021-01-07","cited_by_count":5,"area":null},{"id":"2106.02260","title":"Regularization and Reparameterization Avoid Vanishing Gradients in Sigmoid-Type Networks","created":"2021-06-04","cited_by_count":5,"area":null},{"id":"2109.08958","title":"AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks","created":"2021-09-18","cited_by_count":5,"area":null},{"id":"2205.08385","title":"Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs","created":"2022-05-12","cited_by_count":5,"area":null},{"id":"2210.01245","title":"Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks","created":"2022-10-03","cited_by_count":5,"area":null},{"id":"2212.05599","title":"Orthogonal SVD Covariance Conditioning and Latent Disentanglement","created":"2022-12-11","cited_by_count":5,"area":null},{"id":"2308.12243","title":"Multi-Objective Optimization for Sparse Deep Multi-Task Learning","created":"2023-08-23","cited_by_count":5,"area":null},{"id":"2410.02242","title":"Robust Weight Initialization for Tanh Neural Networks with Fixed Point Analysis","created":"2024-10-03","cited_by_count":5,"area":null},{"id":"2410.05500","title":"Residual Kolmogorov-Arnold Network for Enhanced Deep Learning","created":"2024-10-07","cited_by_count":5,"area":null},{"id":"2505.12909","title":"Sinusoidal Initialization, Time for a New Start","created":"2025-05-19","cited_by_count":5,"area":null},{"id":"2206.01261","title":"Entangled Residual Mappings","created":"2022-06-02","cited_by_count":4,"area":null},{"id":"2207.02119","title":"Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality","created":"2022-07-05","cited_by_count":4,"area":null},{"id":"2212.13970","title":"Breaking the Architecture Barrier: A Method for Efficient Knowledge Transfer Across Networks","created":"2022-12-28","cited_by_count":4,"area":null},{"id":"2304.01620","title":"Image Blind Denoising Using Dual Convolutional Neural Network with Skip Connection","created":"2023-04-04","cited_by_count":4,"area":null},{"id":"2312.02622","title":"On the Initialization of Graph Neural Networks","created":"2023-12-05","cited_by_count":4,"area":null},{"id":"2401.10178","title":"Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields","created":"2024-01-18","cited_by_count":4,"area":null},{"id":"2103.04331","title":"Auto-tuning of Deep Neural Networks by Conflicting Layer Removal","created":"2021-03-07","cited_by_count":3,"area":null},{"id":"2105.10335","title":"Data-driven Weight Initialization with Sylvester Solvers","created":"2021-05-02","cited_by_count":3,"area":null},{"id":"2110.09374","title":"Ortho-Shot: Low Displacement Rank Regularization with Data Augmentation for Few-Shot Learning","created":"2021-10-18","cited_by_count":3,"area":null},{"id":"2202.00834","title":"Nonlinear Initialization Methods for Low-Rank Neural Networks","created":"2022-02-02","cited_by_count":3,"area":null},{"id":"2206.13568","title":"AutoInit: Automatic Initialization via Jacobian Tuning","created":"2022-06-27","cited_by_count":3,"area":null},{"id":"2302.11446","title":"Singular value decomposition based matrix surgery","created":"2023-02-22","cited_by_count":3,"area":null},{"id":"2304.05600","title":"Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning","created":"2023-04-12","cited_by_count":3,"area":null},{"id":"2506.08764","title":"On the Stability of the Jacobian Matrix in Deep Neural Networks","created":"2025-06-10","cited_by_count":3,"area":null},{"id":"2508.01519","title":"The Vanishing Gradient Problem for Stiff Neural Differential Equations","created":"2025-08-02","cited_by_count":3,"area":null},{"id":"2101.06848","title":"Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations","created":"2021-01-18","cited_by_count":2,"area":null},{"id":"2105.05916","title":"Dynamical Isometry: The Missing Ingredient for Neural Network Pruning","created":"2021-05-12","cited_by_count":2,"area":null},{"id":"2106.12746","title":"A Global Appearance and Local Coding Distortion based Fusion Framework for CNN based Filtering in Video Coding","created":"2021-06-24","cited_by_count":2,"area":null},{"id":"2109.00125","title":"A Weight Initialization Based on the Linear Product Structure for Neural Networks","created":"2021-09-01","cited_by_count":2,"area":null},{"id":"2212.12086","title":"Eigenvalue initialisation and regularisation for Koopman autoencoders","created":"2022-12-23","cited_by_count":2,"area":null},{"id":"2307.14823","title":"Fading memory as inductive bias in residual recurrent networks","created":"2023-07-27","cited_by_count":2,"area":null},{"id":"2501.19114","title":"Principal Components for Neural Network Initialization","created":"2025-01-31","cited_by_count":2,"area":null},{"id":"2509.25979","title":"Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifier","created":"2025-09-30","cited_by_count":2,"area":null},{"id":"2606.27880","title":"OrthoTryOn: Geometric Orthogonalization for Conflict-Free Unified Fashion Generation","created":"2026-06-26","cited_by_count":2,"area":null},{"id":"2103.12499","title":"Initializing ReLU networks in an expressive subspace of weights","created":"2021-03-23","cited_by_count":1,"area":null},{"id":"2111.14788","title":"Comparing Machine Learning and Interpolation Methods for Loop-Level Calculations","created":"2021-11-29","cited_by_count":1,"area":null},{"id":"2207.10049","title":"Pretraining a Neural Network before Knowing Its Architecture","created":"2022-07-20","cited_by_count":1,"area":null},{"id":"2209.06344","title":"CNN-Trans-Enc: A CNN-Enhanced Transformer-Encoder On Top Of Static BERT representations for Document Classification","created":"2022-09-13","cited_by_count":1,"area":null},{"id":"2209.15283","title":"Sparse tree-based initialization for neural networks","created":"2022-09-30","cited_by_count":1,"area":null},{"id":"2302.03193","title":"On the Ideal Number of Groups for Isometric Gradient Propagation","created":"2023-02-07","cited_by_count":1,"area":null},{"id":"2311.13722","title":"Deep Learning as a Method for Inversion of NMR Signals","created":"2023-11-22","cited_by_count":1,"area":null},{"id":"2311.14658","title":"Convergence Analysis for Learning Orthonormal Deep Linear Neural Networks","created":"2023-11-24","cited_by_count":1,"area":null},{"id":"2406.00348","title":"An Effective Weight Initialization Method for Deep Learning: Application to Satellite Image Classification","created":"2024-06-01","cited_by_count":1,"area":null},{"id":"2408.08408","title":"Oja's plasticity rule overcomes several challenges of training neural networks under biological constraints","created":"2024-08-15","cited_by_count":1,"area":null},{"id":"2502.07739","title":"HRP: High-Rank Preheating for Superior LoRA Initialization","created":"2025-02-11","cited_by_count":1,"area":null},{"id":"2508.10064","title":"Dynamical Alignment: A Principle for Adaptive Neural Computation","created":"2025-08-13","cited_by_count":1,"area":null},{"id":"2510.15012","title":"From Universal Approximation Theorem to Tropical Geometry of Multi-Layer Perceptrons","created":"2025-10-16","cited_by_count":1,"area":null},{"id":"2101.01178","title":"Advances in Electron Microscopy with Deep Learning","created":"2021-01-04","cited_by_count":0,"area":null},{"id":"2104.13050","title":"Confined Gradient Descent: Privacy-preserving Optimization for Federated Learning","created":"2021-04-27","cited_by_count":0,"area":null},{"id":"2106.12479","title":"Classifying Textual Data with Pre-trained Vision Models through Transfer Learning and Data Transformations","created":"2021-06-23","cited_by_count":0,"area":null},{"id":"2107.08471","title":"A stepped sampling method for video detection using LSTM","created":"2021-07-18","cited_by_count":0,"area":null},{"id":"2109.09016","title":"The Unreasonable Effectiveness of the Final Batch Normalization Layer","created":"2021-09-18","cited_by_count":0,"area":null},{"id":"2110.00791","title":"Optimizing Neural Network for Computer Vision task in Edge Device","created":"2021-10-02","cited_by_count":0,"area":null},{"id":"2112.03777","title":"Variance-Aware Weight Initialization for Point Convolutional Neural Networks","created":"2021-12-07","cited_by_count":0,"area":null},{"id":"2201.05217","title":"Learning Enhancement of CNNs via Separation Index Maximizing at the First Convolutional Layer","created":"2022-01-13","cited_by_count":0,"area":null},{"id":"2203.17265","title":"A 23 MW data centre is all you need","created":"2022-03-31","cited_by_count":0,"area":null},{"id":"2210.02411","title":"Dynamical Isometry for Residual Networks","created":"2022-10-05","cited_by_count":0,"area":null},{"id":"2301.00424","title":"GoogLe2Net: Going Transverse with Convolutions","created":"2023-01-01","cited_by_count":0,"area":null},{"id":"2304.03374","title":"Optimizing Neural Networks through Activation Function Discovery and Automatic Weight Initialization","created":"2023-04-06","cited_by_count":0,"area":null},{"id":"2403.08265","title":"Random Search as a Baseline for Sparse Neural Network Architecture Search","created":"2024-03-13","cited_by_count":0,"area":null},{"id":"2407.19044","title":"Advancing Neural Network Performance through Emergence-Promoting Initialization Scheme","created":"2024-07-26","cited_by_count":0,"area":null},{"id":"2410.00580","title":"Deep activity propagation via weight initialization in spiking neural networks","created":"2024-10-01","cited_by_count":0,"area":null}],"next_cursor":"100"}