{"total":309,"rows":[{"id":"2308.14508","title":"LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding","created":"2023-08-28","cited_by_count":1417,"area":null},{"id":"2305.13048","title":"RWKV: Reinventing RNNs for the Transformer Era","created":"2023-05-22","cited_by_count":827,"area":null},{"id":"2307.08621","title":"Retentive Network: A Successor to Transformer for Large Language Models","created":"2023-07-17","cited_by_count":613,"area":null},{"id":"2412.06464","title":"Gated Delta Networks: Improving Mamba2 with Delta Rule","created":"2024-12-09","cited_by_count":532,"area":null},{"id":"2404.02060","title":"Long-context LLMs Struggle with Long In-context Learning","created":"2024-04-02","cited_by_count":286,"area":null},{"id":"2402.19427","title":"Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models","created":"2024-02-29","cited_by_count":242,"area":null},{"id":"2404.05892","title":"Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence","created":"2024-04-08","cited_by_count":159,"area":null},{"id":"2509.26645","title":"TTT3R: 3D Reconstruction as Test-Time Training","created":"2025-09-30","cited_by_count":129,"area":null},{"id":"2404.07904","title":"HGRN2: Gated Linear RNNs with State Expansion","created":"2024-04-11","cited_by_count":127,"area":null},{"id":"2311.04823","title":"Hierarchically Gated Recurrent Neural Network for Sequence Modeling","created":"2023-11-08","cited_by_count":102,"area":null},{"id":"2401.08092","title":"A Survey of Resource-efficient LLM and Multimodal Foundation Models","created":"2024-01-16","cited_by_count":86,"area":null},{"id":"2306.00946","title":"Exposing Attention Glitches with Flip-Flop Language Modeling","created":"2023-06-01","cited_by_count":75,"area":null},{"id":"2502.10297","title":"DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products","created":"2025-02-14","cited_by_count":75,"area":null},{"id":"2501.12352","title":"Test-time regression: a unifying framework for designing sequence models with associative memory","created":"2025-01-21","cited_by_count":74,"area":null},{"id":"2404.07839","title":"RecurrentGemma: Moving Past Transformers for Efficient Open Language Models","created":"2024-04-11","cited_by_count":64,"area":null},{"id":"2504.13173","title":"It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization","created":"2025-04-17","cited_by_count":64,"area":null},{"id":"2404.09516","title":"State Space Model for New-Generation Network Alternative to Transformers: A Survey","created":"2024-04-15","cited_by_count":59,"area":null},{"id":"2503.02130","title":"Forgetting Transformer: Softmax Attention with a Forget Gate","created":"2025-03-03","cited_by_count":53,"area":null},{"id":"2406.16747","title":"Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers","created":"2024-06-24","cited_by_count":52,"area":null},{"id":"2310.02619","title":"Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs","created":"2023-10-04","cited_by_count":49,"area":null},{"id":"2502.13685","title":"MoM: Linear Sequence Modeling with Mixture-of-Memories","created":"2025-02-19","cited_by_count":47,"area":null},{"id":"2311.01927","title":"GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling","created":"2023-11-03","cited_by_count":46,"area":null},{"id":"2405.17394","title":"The Expressive Capacity of State Space Models: A Formal Language Perspective","created":"2024-05-27","cited_by_count":46,"area":null},{"id":"2509.26030","title":"Muon Outperforms Adam in Tail-End Associative Memory Learning","created":"2025-09-30","cited_by_count":46,"area":null},{"id":"2309.12252","title":"Parallelizing non-linear sequential models over the sequence length","created":"2023-09-21","cited_by_count":45,"area":null},{"id":"2502.10818","title":"On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning","created":"2025-02-15","cited_by_count":43,"area":null},{"id":"2309.13414","title":"State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory","created":"2023-09-23","cited_by_count":39,"area":null},{"id":"2410.03943","title":"Oscillatory State-Space Models","created":"2024-10-04","cited_by_count":39,"area":null},{"id":"2407.19115","title":"Towards Scalable and Stable Parallelization of Nonlinear RNNs","created":"2024-07-26","cited_by_count":38,"area":null},{"id":"2410.01201","title":"Were RNNs All We Needed?","created":"2024-10-02","cited_by_count":38,"area":null},{"id":"2402.18512","title":"Log Neural Controlled Differential Equations: The Lie Brackets Make a Difference","created":"2024-02-28","cited_by_count":35,"area":null},{"id":"2307.14995","title":"TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer","created":"2023-07-27","cited_by_count":34,"area":null},{"id":"2310.02980","title":"Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors","created":"2023-10-04","cited_by_count":34,"area":null},{"id":"2404.18508","title":"Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Models","created":"2024-04-29","cited_by_count":31,"area":null},{"id":"2405.17381","title":"Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention","created":"2024-05-27","cited_by_count":31,"area":null},{"id":"2409.02313","title":"On the Benefits of Memory for Modeling Time-Dependent PDEs","created":"2024-09-03","cited_by_count":31,"area":null},{"id":"2311.14495","title":"StableSSM: Alleviating the Curse of Memory in State-space Models through Stable Reparameterization","created":"2023-11-24","cited_by_count":30,"area":null},{"id":"2406.09347","title":"Separations in the Representational Capabilities of Transformers and Recurrent Architectures","created":"2024-06-13","cited_by_count":29,"area":null},{"id":"2606.18206","title":"Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers","created":"2026-06-16","cited_by_count":28,"area":null},{"id":"2405.15731","title":"Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks","created":"2024-05-24","cited_by_count":26,"area":null},{"id":"2407.06324","title":"B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory","created":"2024-07-08","cited_by_count":26,"area":null},{"id":"2305.15947","title":"Online learning of long-range dependencies","created":"2023-05-25","cited_by_count":25,"area":null},{"id":"2405.21064","title":"Recurrent neural networks: vanishing and exploding gradients are not the end of the story","created":"2024-05-31","cited_by_count":25,"area":null},{"id":"2406.02528","title":"Scalable MatMul-free Language Modeling","created":"2024-06-04","cited_by_count":25,"area":null},{"id":"2310.18780","title":"Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions","created":"2023-10-28","cited_by_count":24,"area":null},{"id":"2402.04875","title":"On Provable Length and Compositional Generalization","created":"2024-02-07","cited_by_count":22,"area":null},{"id":"2312.06837","title":"Spectral State Space Models","created":"2023-12-11","cited_by_count":21,"area":null},{"id":"2411.10741","title":"MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map","created":"2024-11-16","cited_by_count":21,"area":null},{"id":"2305.19044","title":"Exploring the Promise and Limits of Real-Time Recurrent Learning","created":"2023-05-30","cited_by_count":19,"area":null},{"id":"2401.00955","title":"Learning Long Sequences in Spiking Neural Networks","created":"2023-12-14","cited_by_count":19,"area":null},{"id":"2403.16899","title":"State Space Models as Foundation Models: A Control Theoretic Overview","created":"2024-03-25","cited_by_count":19,"area":null},{"id":"2406.16690","title":"Scaling Laws for Linear Complexity Language Models","created":"2024-06-24","cited_by_count":19,"area":null},{"id":"2412.18603","title":"Long-Form Speech Generation with Spoken Language Models","created":"2024-12-24","cited_by_count":19,"area":null},{"id":"2504.05646","title":"Lattice: Learning to Efficiently Compress the Memory","created":"2025-04-08","cited_by_count":19,"area":null},{"id":"2310.01698","title":"Robustifying State-space Models for Long Sequences via Approximate Diagonalization","created":"2023-10-02","cited_by_count":18,"area":null},{"id":"2405.06147","title":"State-Free Inference of State-Space Models: The Transfer Function Approach","created":"2024-05-10","cited_by_count":18,"area":null},{"id":"2505.17761","title":"Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models","created":"2025-05-23","cited_by_count":18,"area":null},{"id":"2402.08132","title":"On the Resurgence of Recurrent Models for Long Sequences -- Survey and Research Opportunities in the Transformer Era","created":"2024-02-12","cited_by_count":17,"area":null},{"id":"2410.02035","title":"Tuning Frequency Bias of State Space Models","created":"2024-10-02","cited_by_count":17,"area":null},{"id":"2410.03464","title":"S7: Selective and Simplified State Space Layers for Sequence Modeling","created":"2024-10-04","cited_by_count":17,"area":null},{"id":"2306.04532","title":"Long Sequence Hopfield Memory","created":"2023-06-07","cited_by_count":16,"area":null},{"id":"2309.08045","title":"Traveling Waves Encode the Recent Past and Enhance Sequence Learning","created":"2023-09-03","cited_by_count":16,"area":null},{"id":"2405.20799","title":"Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patching","created":"2024-05-31","cited_by_count":16,"area":null},{"id":"2406.00943","title":"State Space Models on Temporal Graphs: A First-Principles Study","created":"2024-06-03","cited_by_count":16,"area":null},{"id":"2509.22284","title":"Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models","created":"2025-09-26","cited_by_count":16,"area":null},{"id":"2406.16749","title":"Inferring stochastic low-rank recurrent neural networks from neural data","created":"2024-06-24","cited_by_count":15,"area":null},{"id":"2506.05584","title":"TabFlex: Scaling Tabular Learning to Millions with Linear Attention","created":"2025-06-05","cited_by_count":15,"area":null},{"id":"2510.21450","title":"ParaRNN: Unlocking Parallel Training of Nonlinear RNNs for Large Language Models","created":"2025-10-24","cited_by_count":15,"area":null},{"id":"2305.03796","title":"Transformer Working Memory Enables Regular Language Reasoning and Natural Language Length Extrapolation","created":"2023-05-05","cited_by_count":13,"area":null},{"id":"2309.07412","title":"Advancing Regular Language Reasoning in Linear Recurrent Neural Networks","created":"2023-09-14","cited_by_count":13,"area":null},{"id":"2503.14376","title":"Tiled Flash Linear Attention: More Efficient Linear RNN and xLSTM Kernels","created":"2025-03-18","cited_by_count":13,"area":null},{"id":"2507.16577","title":"Scaling Linear Attention with Sparse State Expansion","created":"2025-07-22","cited_by_count":13,"area":null},{"id":"2602.21204","title":"Test-Time Training with KV Binding Is Secretly Linear Attention","created":"2026-02-24","cited_by_count":13,"area":null},{"id":"2305.14952","title":"Focus Your Attention (with Adaptive IIR Filters)","created":"2023-05-24","cited_by_count":12,"area":null},{"id":"2305.19370","title":"Blockwise Parallel Transformer for Large Context Models","created":"2023-05-30","cited_by_count":12,"area":null},{"id":"2405.13975","title":"HOPE for a Robust Parameterization of Long-memory State Space Models","created":"2024-05-22","cited_by_count":12,"area":null},{"id":"2410.14195","title":"Rethinking Transformer for Long Contextual Histopathology Whole Slide Image Analysis","created":"2024-10-18","cited_by_count":12,"area":null},{"id":"2503.01868","title":"Systems and Algorithms for Convolutional Multi-Hybrid Language Models at Scale","created":"2025-02-25","cited_by_count":12,"area":null},{"id":"2504.04308","title":"Gating is Weighting: Understanding Gated Linear Attention through In-context Learning","created":"2025-04-06","cited_by_count":12,"area":null},{"id":"2508.06097","title":"Recurrent Deep Differentiable Logic Gate Networks","created":"2025-08-08","cited_by_count":12,"area":null},{"id":"2305.19190","title":"Inverse Approximation Theory for Nonlinear Recurrent Neural Networks","created":"2023-05-30","cited_by_count":11,"area":null},{"id":"2312.01538","title":"Recurrent Distance Filtering for Graph Representation Learning","created":"2023-12-03","cited_by_count":11,"area":null},{"id":"2402.12138","title":"Perceiving Longer Sequences With Bi-Directional Cross-Attention Transformers","created":"2024-02-19","cited_by_count":11,"area":null},{"id":"2405.02670","title":"From Generalization Analysis to Optimization Designs for State Space Models","created":"2024-05-04","cited_by_count":11,"area":null},{"id":"2412.13328","title":"Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models","created":"2024-12-17","cited_by_count":11,"area":null},{"id":"2505.14172","title":"The Strawberry Problem: Emergence of Character-level Understanding in Tokenized Language Models","created":"2025-05-20","cited_by_count":11,"area":null},{"id":"2511.21016","title":"Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regression","created":"2025-11-26","cited_by_count":11,"area":null},{"id":"2311.16620","title":"On the Long Range Abilities of Transformers","created":"2023-11-28","cited_by_count":10,"area":null},{"id":"2403.16933","title":"Backpropagation through space, time, and the brain","created":"2024-03-25","cited_by_count":10,"area":null},{"id":"2406.09477","title":"Q-S5: Towards Quantized State Space Models","created":"2024-06-13","cited_by_count":10,"area":null},{"id":"2407.06312","title":"Adversarial dynamical systems characterize when data-driven learning succeeds or fails","created":"2024-07-08","cited_by_count":10,"area":null},{"id":"2505.18728","title":"Message-Passing State-Space Models: Improving Graph Learning with Modern Sequence Modeling","created":"2025-05-24","cited_by_count":10,"area":null},{"id":"2405.21022","title":"You Only Scan Once: Efficient Multi-dimension Sequential Modeling with LightNet","created":"2024-05-31","cited_by_count":9,"area":null},{"id":"2407.08112","title":"How Well Can a Long Sequence Model Model Long Sequences? Comparing Architechtural Inductive Biases on Long-Context Abilities","created":"2024-07-11","cited_by_count":9,"area":null},{"id":"2505.12171","title":"Learning to Dissipate Energy in Oscillatory State-Space Models","created":"2025-05-17","cited_by_count":9,"area":null},{"id":"2511.17388","title":"Selective Rotary Position Embedding","created":"2025-11-21","cited_by_count":9,"area":null},{"id":"2311.16943","title":"Image segmentation with traveling waves in an exactly solvable recurrent neural network","created":"2023-11-28","cited_by_count":8,"area":null},{"id":"2505.19959","title":"MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models","created":"2025-05-26","cited_by_count":8,"area":null},{"id":"2508.08435","title":"Fast weight programming and linear transformers: from machine learning to neurobiology","created":"2025-08-11","cited_by_count":8,"area":null},{"id":"2508.19029","title":"Revisiting associative recall in modern recurrent models","created":"2025-08-26","cited_by_count":8,"area":null}],"next_cursor":"100"}