{"total":129,"rows":[{"id":"2306.15595","title":"Extending Context Window of Large Language Models via Positional Interpolation","created":"2023-06-27","cited_by_count":740,"area":null},{"id":"2304.14767","title":"Dissecting Recall of Factual Associations in Auto-Regressive Language Models","created":"2023-04-28","cited_by_count":528,"area":null},{"id":"2305.19466","title":"The Impact of Positional Encoding on Length Generalization in Transformers","created":"2023-05-31","cited_by_count":327,"area":null},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268,"area":null},{"id":"2305.16300","title":"Landmark Attention: Random-Access Infinite Context Length for Transformers","created":"2023-05-25","cited_by_count":213,"area":null},{"id":"2307.03170","title":"Focused Transformer: Contrastive Training for Context Scaling","created":"2023-07-06","cited_by_count":133,"area":null},{"id":"2410.06205","title":"Round and Round We Go! What makes Rotary Positional Encodings useful?","created":"2024-10-08","cited_by_count":115,"area":null},{"id":"2309.10400","title":"PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training","created":"2023-09-19","cited_by_count":111,"area":null},{"id":"2303.11504","title":"Language Model Behavior: A Comprehensive Survey","created":"2023-03-20","cited_by_count":92,"area":null},{"id":"2309.04827","title":"Neurons in Large Language Models: Dead, N-gram, Positional","created":"2023-09-09","cited_by_count":88,"area":null},{"id":"2402.09371","title":"Transformers Can Achieve Length Generalization But Not Robustly","created":"2024-02-14","cited_by_count":86,"area":null},{"id":"2306.04891","title":"In-Context Learning through the Bayesian Prism","created":"2023-06-08","cited_by_count":83,"area":null},{"id":"2306.00946","title":"Exposing Attention Glitches with Flip-Flop Language Modeling","created":"2023-06-01","cited_by_count":75,"area":null},{"id":"2405.18719","title":"Contextual Position Encoding: Learning to Count What's Important","created":"2024-05-29","cited_by_count":54,"area":null},{"id":"2404.12224","title":"Length Generalization of Causal Transformers without Position Encoding","created":"2024-04-18","cited_by_count":47,"area":null},{"id":"2501.18795","title":"Rope to Nope and Back Again: A New Hybrid Attention Strategy","created":"2025-01-30","cited_by_count":47,"area":null},{"id":"2502.01563","title":"Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding","created":"2025-02-03","cited_by_count":47,"area":null},{"id":"2411.03766","title":"Number Cookbook: Number Understanding of Language Models and How to Improve It","created":"2024-11-06","cited_by_count":42,"area":null},{"id":"2403.18167","title":"Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations","created":"2024-03-27","cited_by_count":38,"area":null},{"id":"2410.24087","title":"In-Context Fine-Tuning for Time-Series Foundation Models","created":"2024-10-31","cited_by_count":35,"area":null},{"id":"2401.07872","title":"The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey","created":"2024-01-15","cited_by_count":33,"area":null},{"id":"2312.17044","title":"Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding","created":"2023-12-28","cited_by_count":32,"area":null},{"id":"2407.09777","title":"Graph Transformers: A Survey","created":"2024-07-13","cited_by_count":32,"area":null},{"id":"2405.14722","title":"DAPE: Data-Adaptive Positional Encoding for Length Extrapolation","created":"2024-05-23","cited_by_count":31,"area":null},{"id":"2402.03902","title":"A phase transition between positional and semantic learning in a solvable model of dot-product attention","created":"2024-02-06","cited_by_count":30,"area":null},{"id":"2207.06983","title":"Multitrack Music Transformer","created":"2022-07-14","cited_by_count":28,"area":null},{"id":"2411.07602","title":"Circuit Complexity Bounds for RoPE-based Transformer Architecture","created":"2024-11-12","cited_by_count":26,"area":null},{"id":"2512.12167","title":"Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings","created":"2025-12-13","cited_by_count":26,"area":null},{"id":"2405.20131","title":"Language Models Need Inductive Biases to Count Inductively","created":"2024-05-30","cited_by_count":25,"area":null},{"id":"2407.03310","title":"Universal Length Generalization with Turing Programs","created":"2024-07-03","cited_by_count":25,"area":null},{"id":"2401.16421","title":"Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation","created":"2024-01-29","cited_by_count":24,"area":null},{"id":"2305.13571","title":"Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings","created":"2023-05-23","cited_by_count":23,"area":null},{"id":"2212.08979","title":"Language model acceptability judgements are not always robust to context","created":"2022-12-18","cited_by_count":22,"area":null},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22,"area":null},{"id":"2210.00400","title":"Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks","created":"2022-10-02","cited_by_count":19,"area":null},{"id":"2504.08719","title":"SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling","created":"2025-04-11","cited_by_count":19,"area":null},{"id":"2311.05613","title":"Window Attention is Bugged: How not to Interpolate Position Embeddings","created":"2023-11-09","cited_by_count":18,"area":null},{"id":"2405.18009","title":"Exploring Context Window of Large Language Models via Decomposed Positional Vectors","created":"2024-05-28","cited_by_count":18,"area":null},{"id":"2210.12574","title":"The Curious Case of Absolute Position Embeddings","created":"2022-10-23","cited_by_count":17,"area":null},{"id":"2410.17980","title":"Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study","created":"2024-10-23","cited_by_count":13,"area":null},{"id":"2506.00744","title":"Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers","created":"2025-05-31","cited_by_count":12,"area":null},{"id":"2410.04798","title":"DAPE V2: Process Attention Score as Feature Map for Length Extrapolation","created":"2024-10-07","cited_by_count":10,"area":null},{"id":"2509.05218","title":"HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models","created":"2025-09-05","cited_by_count":10,"area":null},{"id":"2505.16983","title":"LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding","created":"2025-05-22","cited_by_count":9,"area":null},{"id":"2210.07468","title":"Transparency Helps Reveal When Language Models Learn Meaning","created":"2022-10-14","cited_by_count":8,"area":null},{"id":"2410.13779","title":"The Mystery of the Pathological Path-star Task for Language Models","created":"2024-10-17","cited_by_count":8,"area":null},{"id":"2310.08796","title":"End-to-end Story Plot Generator","created":"2023-10-13","cited_by_count":7,"area":null},{"id":"2312.04556","title":"Large Language Models for Mathematicians","created":"2023-12-07","cited_by_count":7,"area":null},{"id":"2406.11403","title":"Multimodal Structured Generation: CVPR's 2nd MMFM Challenge Technical Report","created":"2024-06-17","cited_by_count":7,"area":null},{"id":"2404.02047","title":"Learning Transactions Representations for Information Management in Banks: Mastering Local, Global, and External Knowledge","created":"2024-04-02","cited_by_count":6,"area":null},{"id":"2409.07154","title":"Recurrent Aggregators in Neural Algorithmic Reasoning","created":"2024-09-11","cited_by_count":6,"area":null},{"id":"2501.00659","title":"Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? Revisiting a Petroglyph","created":"2024-12-31","cited_by_count":6,"area":null},{"id":"2501.06141","title":"Emergent Symbol-like Number Variables in Artificial Neural Networks","created":"2025-01-10","cited_by_count":6,"area":null},{"id":"2504.02827","title":"On Vanishing Variance in Transformer Length Generalization","created":"2025-04-03","cited_by_count":6,"area":null},{"id":"2509.00622","title":"BALM-TSF: Balanced Multimodal Alignment for LLM-Based Time Series Forecasting","created":"2025-08-30","cited_by_count":6,"area":null},{"id":"2204.07555","title":"Chinese Idiom Paraphrasing","created":"2022-04-15","cited_by_count":5,"area":null},{"id":"2209.12816","title":"Fast-FNet: Accelerating Transformer Encoder Models via Efficient Fourier Layers","created":"2022-09-26","cited_by_count":5,"area":null},{"id":"2402.13301","title":"Structure-informed Positional Encoding for Music Generation","created":"2024-02-20","cited_by_count":5,"area":null},{"id":"2505.13027","title":"Deconstructing Positional Information: From Attention Logits to Training Biases","created":"2025-05-19","cited_by_count":5,"area":null},{"id":"2505.19084","title":"Jodi: Unification of Visual Generation and Understanding via Joint Modeling","created":"2025-05-25","cited_by_count":5,"area":null},{"id":"2511.09146","title":"DoPE: Denoising Rotary Position Embedding","created":"2025-11-12","cited_by_count":5,"area":null},{"id":"2512.07805","title":"Group Representational Position Encoding","created":"2025-12-08","cited_by_count":5,"area":null},{"id":"2211.04427","title":"Word Order Matters when you Increase Masking","created":"2022-11-08","cited_by_count":4,"area":null},{"id":"2404.04530","title":"A Morphology-Based Investigation of Positional Encodings","created":"2024-04-06","cited_by_count":4,"area":null},{"id":"2502.06902","title":"Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training","created":"2025-02-09","cited_by_count":4,"area":null},{"id":"2509.03643","title":"CEHR-XGPT: A Scalable Multi-Task Foundation Model for Electronic Health Records","created":"2025-09-03","cited_by_count":4,"area":null},{"id":"2510.09477","title":"Efficient Autoregressive Inference for Transformer Probabilistic Models","created":"2025-10-10","cited_by_count":4,"area":null},{"id":"2601.07988","title":"From Word Sequences to Behavioral Sequences: Adapting Modeling and Evaluation Paradigms for Longitudinal NLP","created":"2026-01-12","cited_by_count":4,"area":null},{"id":"2205.12787","title":"Impartial Games: A Challenge for Reinforcement Learning","created":"2022-05-25","cited_by_count":3,"area":null},{"id":"2311.18194","title":"Positional Information Matters for Invariant In-Context Learning: A Case Study of Simple Function Classes","created":"2023-11-30","cited_by_count":3,"area":null},{"id":"2406.00131","title":"From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When","created":"2024-05-31","cited_by_count":3,"area":null},{"id":"2406.02536","title":"Mitigate Position Bias in Large Language Models via Scaling a Single Dimension","created":"2024-06-04","cited_by_count":3,"area":null},{"id":"2502.08662","title":"RoToR: Towards More Reliable Responses for Order-Invariant Inputs","created":"2025-02-10","cited_by_count":3,"area":null},{"id":"2503.10542","title":"Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More","created":"2025-03-13","cited_by_count":3,"area":null},{"id":"2505.22842","title":"Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation","created":"2025-05-28","cited_by_count":3,"area":null},{"id":"2511.08368","title":"A Circular Argument : Does RoPE need to be Equivariant for Vision?","created":"2025-11-11","cited_by_count":3,"area":null},{"id":"2603.15506","title":"Seeking SOTA: Time-Series Forecasting Must Adopt Taxonomy-Specific Evaluation to Dispel Illusory Gains","created":"2026-03-16","cited_by_count":3,"area":null},{"id":"2309.00857","title":"Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languages","created":"2023-09-02","cited_by_count":2,"area":null},{"id":"2407.20028","title":"Aircraft Trajectory Segmentation-based Contrastive Coding: A Framework for Self-supervised Trajectory Representation","created":"2024-07-29","cited_by_count":2,"area":null},{"id":"2410.15859","title":"Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs","created":"2024-10-21","cited_by_count":2,"area":null},{"id":"2502.10154","title":"Video Soundtrack Generation by Aligning Emotions and Temporal Boundaries","created":"2025-02-14","cited_by_count":2,"area":null},{"id":"2502.10491","title":"F-StrIPE: Fast Structure-Informed Positional Encoding for Symbolic Music Generation","created":"2025-02-14","cited_by_count":2,"area":null},{"id":"2503.08067","title":"Context-aware Biases for Length Extrapolation","created":"2025-03-11","cited_by_count":2,"area":null},{"id":"2504.20980","title":"Jekyll-and-Hyde Tipping Point in an AI's Behavior","created":"2025-04-29","cited_by_count":2,"area":null},{"id":"2506.13277","title":"SeqPE: Transformer with Sequential Position Encoding","created":"2025-06-16","cited_by_count":2,"area":null},{"id":"2602.14100","title":"Character-aware Transformers Learn an Irregular Morphological Pattern Yet None Generalize Like Humans","created":"2026-02-15","cited_by_count":2,"area":null},{"id":"2603.22586","title":"A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks","created":"2026-03-23","cited_by_count":2,"area":null},{"id":"2208.14554","title":"Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?","created":"2022-08-30","cited_by_count":1,"area":null},{"id":"2302.07162","title":"Semiconductor Fab Scheduling with Self-Supervised and Reinforcement Learning","created":"2023-02-14","cited_by_count":1,"area":null},{"id":"2310.12864","title":"The Locality and Symmetry of Positional Encodings","created":"2023-10-19","cited_by_count":1,"area":null},{"id":"2403.13106","title":"Using Shapley interactions to understand how models use structure","created":"2024-03-19","cited_by_count":1,"area":null},{"id":"2407.10949","title":"Representing Rule-based Chatbots with Transformers","created":"2024-07-15","cited_by_count":1,"area":null},{"id":"2411.15372","title":"Transforming NLU with Babylon: A Case Study in Development of Real-time, Edge-Efficient, Multi-Intent Translation System for Automated Drive-Thru Ordering","created":"2024-11-22","cited_by_count":1,"area":null},{"id":"2501.11730","title":"Transformer Vibration Forecasting for Advancing Rail Safety and Maintenance 4.0","created":"2025-01-20","cited_by_count":1,"area":null},{"id":"2502.18823","title":"Evidence-Driven Marker Extraction for Social Media Suicide Risk Detection","created":"2025-02-26","cited_by_count":1,"area":null},{"id":"2509.12635","title":"Positional Encoding via Token-Aware Phase Attention","created":"2025-09-16","cited_by_count":1,"area":null},{"id":"2509.16278","title":"Language Modeling with Learned Meta-Tokens","created":"2025-09-18","cited_by_count":1,"area":null},{"id":"2509.21042","title":"LayerNorm Induces Recency Bias in Transformer Decoders","created":"2025-09-25","cited_by_count":1,"area":null},{"id":"2510.10489","title":"Head-wise Adaptive Rotary Positional Encoding for Fine-Grained Image Generation","created":"2025-10-12","cited_by_count":1,"area":null},{"id":"2511.19705","title":"CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding","created":"2025-11-24","cited_by_count":1,"area":null}],"next_cursor":"100"}