{"total":1003,"rows":[{"id":"2102.12452","title":"Probing Classifiers: Promises, Shortcomings, and Advances","created":"2021-02-24","cited_by_count":876,"area":null},{"id":"2103.10385","title":"GPT Understands, Too","created":"2021-03-18","cited_by_count":716,"area":null},{"id":"2105.13290","title":"CogView: Mastering Text-to-Image Generation via Transformers","created":"2021-05-26","cited_by_count":694,"area":null},{"id":"2104.08696","title":"Knowledge Neurons in Pretrained Transformers","created":"2021-04-18","cited_by_count":601,"area":null},{"id":"2203.14680","title":"Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space","created":"2022-03-28","cited_by_count":532,"area":null},{"id":"2310.01801","title":"Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs","created":"2023-10-03","cited_by_count":474,"area":"inference"},{"id":"2111.01243","title":"Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey","created":"2021-11-01","cited_by_count":463,"area":null},{"id":"2305.07759","title":"TinyStories: How Small Can Language Models Be and Still Speak Coherent English?","created":"2023-05-12","cited_by_count":449,"area":null},{"id":"2106.02997","title":"Causal Abstractions of Neural Networks","created":"2021-06-06","cited_by_count":422,"area":null},{"id":"2105.03824","title":"FNet: Mixing Tokens with Fourier Transforms","created":"2021-05-09","cited_by_count":347,"area":null},{"id":"2104.02112","title":"Efficient Attentions for Long Document Summarization","created":"2021-04-05","cited_by_count":345,"area":null},{"id":"2309.01029","title":"Explainability for Large Language Models: A Survey","created":"2023-09-02","cited_by_count":333,"area":null},{"id":"2106.07139","title":"Pre-Trained Models: Past, Present and Future","created":"2021-06-14","cited_by_count":319,"area":null},{"id":"2106.02636","title":"MERLOT: Multimodal Neural Script Knowledge Models","created":"2021-06-04","cited_by_count":311,"area":null},{"id":"2402.17762","title":"Massive Activations in Large Language Models","created":"2024-02-27","cited_by_count":294,"area":null},{"id":"2410.10819","title":"DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads","created":"2024-10-14","cited_by_count":288,"area":"inference"},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268,"area":null},{"id":"2402.12875","title":"Chain of Thought Empowers Transformers to Solve Inherently Serial Problems","created":"2024-02-20","cited_by_count":260,"area":"lr"},{"id":"2111.12417","title":"N\\\"UWA: Visual Synthesis Pre-training for Neural visUal World creAtion","created":"2021-11-24","cited_by_count":253,"area":null},{"id":"2210.04885","title":"What the DAAM: Interpreting Stable Diffusion Using Cross Attention","created":"2022-10-10","cited_by_count":236,"area":"diffusion"},{"id":"2407.11550","title":"Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference","created":"2024-07-16","cited_by_count":235,"area":"inference"},{"id":"2402.05162","title":"Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications","created":"2024-02-07","cited_by_count":231,"area":"alignment"},{"id":"2312.12148","title":"Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment","created":"2023-12-19","cited_by_count":230,"area":null},{"id":"2403.17887","title":"The Unreasonable Ineffectiveness of the Deeper Layers","created":"2024-03-26","cited_by_count":226,"area":"lr"},{"id":"2202.08791","title":"cosFormer: Rethinking Softmax in Attention","created":"2022-02-17","cited_by_count":213,"area":"lr"},{"id":"2301.04709","title":"Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability","created":"2023-01-11","cited_by_count":205,"area":"mechinterp"},{"id":"2305.00586","title":"How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model","created":"2023-04-30","cited_by_count":202,"area":null},{"id":"2107.00651","title":"AutoFormer: Searching Transformers for Visual Recognition","created":"2021-07-01","cited_by_count":188,"area":null},{"id":"2106.06981","title":"Thinking Like Transformers","created":"2021-06-13","cited_by_count":184,"area":null},{"id":"2106.00737","title":"Implicit Representations of Meaning in Neural Language Models","created":"2021-06-01","cited_by_count":180,"area":null},{"id":"2103.14675","title":"Synthesis of Compositional Animations from Textual Descriptions","created":"2021-03-26","cited_by_count":178,"area":null},{"id":"2403.15412","title":"Towards Measuring and Modeling \"Culture\" in LLMs: A Survey","created":"2024-03-05","cited_by_count":167,"area":null},{"id":"2104.08253","title":"Condenser: a Pre-training Architecture for Dense Retrieval","created":"2021-04-16","cited_by_count":162,"area":null},{"id":"2204.06031","title":"A Review on Language Models as Knowledge Bases","created":"2022-04-12","cited_by_count":157,"area":null},{"id":"2108.04840","title":"Post-hoc Interpretability for Neural NLP: A Survey","created":"2021-08-10","cited_by_count":156,"area":null},{"id":"2306.12929","title":"Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing","created":"2023-06-22","cited_by_count":151,"area":null},{"id":"2110.10090","title":"Inductive Biases and Variable Creation in Self-Attention Mechanisms","created":"2021-10-19","cited_by_count":144,"area":null},{"id":"2407.07071","title":"Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps","created":"2024-07-09","cited_by_count":142,"area":null},{"id":"2109.12948","title":"Understanding and Overcoming the Challenges of Efficient Transformer Quantization","created":"2021-09-27","cited_by_count":141,"area":null},{"id":"2403.03431","title":"Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing","created":"2024-03-06","cited_by_count":135,"area":"diffusion"},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133,"area":null},{"id":"2209.02535","title":"Analyzing Transformers in Embedding Space","created":"2022-09-06","cited_by_count":132,"area":null},{"id":"2209.11326","title":"Towards Faithful Model Explanation in NLP: A Survey","created":"2022-09-22","cited_by_count":130,"area":null},{"id":"2207.13243","title":"Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks","created":"2022-07-27","cited_by_count":129,"area":null},{"id":"2305.01625","title":"Unlimiformer: Long-Range Transformers with Unlimited Length Input","created":"2023-05-02","cited_by_count":128,"area":null},{"id":"2504.02732","title":"Why do LLMs attend to the first token?","created":"2025-04-03","cited_by_count":127,"area":null},{"id":"2102.08473","title":"COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining","created":"2021-02-16","cited_by_count":126,"area":null},{"id":"2309.07311","title":"Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs","created":"2023-09-13","cited_by_count":124,"area":null},{"id":"2401.06104","title":"Transformers are Multi-State RNNs","created":"2024-01-11","cited_by_count":120,"area":"inference"},{"id":"2402.05602","title":"AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers","created":"2024-02-08","cited_by_count":120,"area":"lr"},{"id":"2205.13543","title":"Revealing the Dark Secrets of Masked Image Modeling","created":"2022-05-26","cited_by_count":117,"area":null},{"id":"2108.13888","title":"Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning","created":"2021-08-31","cited_by_count":113,"area":null},{"id":"2402.01761","title":"Rethinking Interpretability in the Era of Large Language Models","created":"2024-01-30","cited_by_count":111,"area":"lr"},{"id":"2112.00826","title":"Inducing Causal Structure for Interpretable Neural Networks","created":"2021-12-01","cited_by_count":110,"area":null},{"id":"2206.04301","title":"Unveiling Transformers with LEGO: a synthetic reasoning task","created":"2022-06-09","cited_by_count":106,"area":null},{"id":"2305.08809","title":"Interpretability at Scale: Identifying Causal Mechanisms in Alpaca","created":"2023-05-15","cited_by_count":103,"area":null},{"id":"2401.04700","title":"Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue","created":"2024-01-09","cited_by_count":103,"area":null},{"id":"2105.09996","title":"VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding","created":"2021-05-20","cited_by_count":102,"area":null},{"id":"2406.16008","title":"Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization","created":"2024-06-23","cited_by_count":101,"area":null},{"id":"2410.11414","title":"ReDeEP: Detecting Hallucination in Retrieval-Augmented Generation via Mechanistic Interpretability","created":"2024-10-15","cited_by_count":101,"area":"rag"},{"id":"2303.11504","title":"Language Model Behavior: A Comprehensive Survey","created":"2023-03-20","cited_by_count":92,"area":null},{"id":"2311.02262","title":"Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs","created":"2023-11-03","cited_by_count":91,"area":null},{"id":"2309.04827","title":"Neurons in Large Language Models: Dead, N-gram, Positional","created":"2023-09-09","cited_by_count":88,"area":null},{"id":"2104.02096","title":"Compressing Visual-linguistic Model via Knowledge Distillation","created":"2021-04-05","cited_by_count":86,"area":"distill"},{"id":"2303.04245","title":"How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding","created":"2023-03-07","cited_by_count":86,"area":null},{"id":"2503.06287","title":"Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding","created":"2025-03-08","cited_by_count":83,"area":"mllm"},{"id":"2402.09268","title":"Transformers, parallel computation, and logarithmic depth","created":"2024-02-14","cited_by_count":79,"area":null},{"id":"2402.17700","title":"RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations","created":"2024-02-27","cited_by_count":79,"area":"mechinterp"},{"id":"2302.13942","title":"Inseq: An Interpretability Toolkit for Sequence Generation Models","created":"2023-02-27","cited_by_count":78,"area":null},{"id":"2103.13084","title":"Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases","created":"2021-03-24","cited_by_count":77,"area":null},{"id":"2110.07736","title":"Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models","created":"2021-10-14","cited_by_count":76,"area":null},{"id":"2406.15765","title":"Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration","created":"2024-06-22","cited_by_count":74,"area":null},{"id":"2104.07885","title":"Probing Across Time: What Does RoBERTa Know and When?","created":"2021-04-16","cited_by_count":73,"area":null},{"id":"2406.12335","title":"Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matters","created":"2024-06-18","cited_by_count":73,"area":"inference"},{"id":"2410.13708","title":"On the Role of Attention Heads in Large Language Model Safety","created":"2024-10-17","cited_by_count":72,"area":"safety"},{"id":"2110.07143","title":"bert2BERT: Towards Reusable Pretrained Language Models","created":"2021-10-14","cited_by_count":70,"area":null},{"id":"2106.12066","title":"It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning","created":"2021-06-22","cited_by_count":69,"area":null},{"id":"2211.01452","title":"MPCFormer: fast, performant and private Transformer inference with MPC","created":"2022-11-02","cited_by_count":69,"area":null},{"id":"2411.17491","title":"What's in the Image? A Deep-Dive into the Vision of Vision Language Models","created":"2024-11-26","cited_by_count":68,"area":"mllm"},{"id":"2309.15098","title":"Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models","created":"2023-09-26","cited_by_count":67,"area":null},{"id":"2111.07970","title":"Triggerless Backdoor Attack for NLP Tasks with Clean Labels","created":"2021-11-15","cited_by_count":66,"area":"safety"},{"id":"2211.07349","title":"Finding Skill Neurons in Pre-trained Transformer-based Language Models","created":"2022-11-14","cited_by_count":66,"area":null},{"id":"2203.04212","title":"Measuring the Mixing of Contextual Information in the Transformer","created":"2022-03-08","cited_by_count":65,"area":null},{"id":"2104.03514","title":"Low-Complexity Probing via Finding Subnetworks","created":"2021-04-08","cited_by_count":64,"area":null},{"id":"2401.12143","title":"Anisotropy Is Inherent to Self-Attention in Transformers","created":"2024-01-22","cited_by_count":63,"area":null},{"id":"2405.10637","title":"Layer-Condensed KV Cache for Efficient Inference of Large Language Models","created":"2024-05-17","cited_by_count":59,"area":"inference"},{"id":"2203.09192","title":"Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Lists","created":"2022-03-17","cited_by_count":55,"area":null},{"id":"2205.11380","title":"Outliers Dimensions that Disrupt Transformers Are Driven by Frequency","created":"2022-05-23","cited_by_count":55,"area":null},{"id":"2205.14140","title":"CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behavior","created":"2022-05-27","cited_by_count":55,"area":null},{"id":"2203.17247","title":"VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformers","created":"2022-03-30","cited_by_count":53,"area":null},{"id":"2402.10350","title":"Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review","created":"2024-02-15","cited_by_count":53,"area":null},{"id":"2406.16747","title":"Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers","created":"2024-06-24","cited_by_count":52,"area":null},{"id":"2412.02104","title":"Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey","created":"2024-12-03","cited_by_count":52,"area":"mllm"},{"id":"2103.16775","title":"Attention, please! A survey of Neural Attention Models in Deep Learning","created":"2021-03-31","cited_by_count":51,"area":null},{"id":"2106.12566","title":"Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding","created":"2021-06-23","cited_by_count":51,"area":null},{"id":"2109.07152","title":"Incorporating Residual and Normalization Layers into Analysis of Masked Language Models","created":"2021-09-15","cited_by_count":51,"area":null},{"id":"2205.00619","title":"POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection","created":"2022-05-02","cited_by_count":51,"area":null},{"id":"2106.08367","title":"What Context Features Can Transformer Language Models Use?","created":"2021-06-15","cited_by_count":50,"area":null},{"id":"2202.06840","title":"What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Code","created":"2022-02-14","cited_by_count":50,"area":null},{"id":"2305.03210","title":"AttentionViz: A Global View of Transformer Attention","created":"2023-05-04","cited_by_count":50,"area":null}],"next_cursor":"100"}