{"total":177,"rows":[{"id":"2210.01936","title":"When and why vision-language models behave like bags-of-words, and what to do about it?","created":"2022-10-04","cited_by_count":610,"area":"mllm"},{"id":"2204.03162","title":"Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality","created":"2022-04-07","cited_by_count":529,"area":"lr"},{"id":"2105.11447","title":"True Few-Shot Learning with Language Models","created":"2021-05-24","cited_by_count":344,"area":null},{"id":"2203.16634","title":"Transformer Language Models without Positional Encodings Still Learn Positional Information","created":"2022-03-30","cited_by_count":150,"area":null},{"id":"2202.07206","title":"Impact of Pretraining Term Frequencies on Few-Shot Reasoning","created":"2022-02-15","cited_by_count":142,"area":"lr"},{"id":"2106.06087","title":"Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models","created":"2021-06-10","cited_by_count":131,"area":null},{"id":"2202.08005","title":"Should You Mask 15% in Masked Language Modeling?","created":"2022-02-16","cited_by_count":118,"area":null},{"id":"2210.03050","title":"State-of-the-art generalisation research in NLP: A taxonomy and review","created":"2022-10-06","cited_by_count":98,"area":null},{"id":"2303.11504","title":"Language Model Behavior: A Comprehensive Survey","created":"2023-03-20","cited_by_count":92,"area":null},{"id":"2311.11829","title":"System 2 Attention (is something you might need too)","created":"2023-11-20","cited_by_count":89,"area":null},{"id":"2208.11857","title":"Shortcut Learning of Large Language Models in Natural Language Understanding","created":"2022-08-25","cited_by_count":85,"area":null},{"id":"2106.09226","title":"Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning","created":"2021-06-17","cited_by_count":82,"area":null},{"id":"2403.19827","title":"Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs","created":"2024-03-28","cited_by_count":65,"area":null},{"id":"2204.08831","title":"Probing for the Usage of Grammatical Number","created":"2022-04-19","cited_by_count":59,"area":null},{"id":"2312.02439","title":"Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation","created":"2023-12-05","cited_by_count":56,"area":"lr"},{"id":"2306.01879","title":"Revisiting the Role of Language Priors in Vision-Language Models","created":"2023-06-02","cited_by_count":50,"area":"mllm"},{"id":"2403.13799","title":"Reverse Training to Nurse the Reversal Curse","created":"2024-03-20","cited_by_count":49,"area":null},{"id":"2109.14989","title":"Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations","created":"2021-09-30","cited_by_count":44,"area":null},{"id":"2211.00768","title":"Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality","created":"2022-11-01","cited_by_count":44,"area":null},{"id":"2302.09466","title":"RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions","created":"2023-02-19","cited_by_count":43,"area":null},{"id":"2310.08753","title":"CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models","created":"2023-10-12","cited_by_count":43,"area":"lr"},{"id":"2410.09918","title":"Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces","created":"2024-10-13","cited_by_count":43,"area":"lr"},{"id":"2406.19470","title":"Changing Answer Order Can Decrease MMLU Accuracy","created":"2024-06-27","cited_by_count":42,"area":null},{"id":"2203.10995","title":"Word Order Does Matter (And Shuffled Language Models Know It)","created":"2022-03-21","cited_by_count":32,"area":null},{"id":"2401.06416","title":"Mission: Impossible Language Models","created":"2024-01-12","cited_by_count":32,"area":null},{"id":"2109.04953","title":"Does Pretraining for Summarization Require Knowledge Transfer?","created":"2021-09-10","cited_by_count":30,"area":null},{"id":"2110.02782","title":"How BPE Affects Memorization in Transformers","created":"2021-10-06","cited_by_count":29,"area":null},{"id":"2203.10326","title":"Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models","created":"2022-03-19","cited_by_count":28,"area":null},{"id":"2305.10588","title":"A Better Way to Do Masked Language Model Scoring","created":"2023-05-17","cited_by_count":27,"area":null},{"id":"2206.10139","title":"Insights into Pre-training via Simpler Synthetic Tasks","created":"2022-06-21","cited_by_count":26,"area":null},{"id":"2109.03537","title":"On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets","created":"2021-09-08","cited_by_count":25,"area":null},{"id":"2210.15615","title":"ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics","created":"2022-10-27","cited_by_count":25,"area":null},{"id":"2406.05183","title":"The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More","created":"2024-06-07","cited_by_count":25,"area":null},{"id":"2203.06204","title":"When classifying grammatical role, BERT doesn't care about word order... except when it matters","created":"2022-03-11","cited_by_count":24,"area":null},{"id":"2207.07611","title":"Position Prediction as an Effective Pretraining Strategy","created":"2022-07-15","cited_by_count":24,"area":null},{"id":"2106.01950","title":"The Case for Translation-Invariant Self-Attention in Transformer-Based Language Models","created":"2021-06-03","cited_by_count":23,"area":null},{"id":"2208.10244","title":"Unit Testing for Concepts in Neural Networks","created":"2022-07-28","cited_by_count":23,"area":null},{"id":"2212.12043","title":"When are Lemons Purple? The Concept Association Bias of Vision-Language Models","created":"2022-12-22","cited_by_count":23,"area":"mllm"},{"id":"2305.13571","title":"Latent Positional Information is in the Self-Attention Variance of Transformer Language Models Without Positional Embeddings","created":"2023-05-23","cited_by_count":23,"area":null},{"id":"2112.05843","title":"Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity","created":"2021-12-10","cited_by_count":22,"area":null},{"id":"2308.12898","title":"Can Linguistic Knowledge Improve Multimodal Alignment in Vision-Language Pretraining?","created":"2023-08-24","cited_by_count":21,"area":"mllm"},{"id":"2205.10226","title":"Do Transformer Models Show Similar Attention Patterns to Task-Specific Human Gaze?","created":"2022-04-25","cited_by_count":20,"area":null},{"id":"2209.14389","title":"Downstream Datasets Make Surprisingly Good Pretraining Corpora","created":"2022-09-28","cited_by_count":20,"area":null},{"id":"2211.00974","title":"Processing Long Legal Documents with Pre-trained Transformers: Modding LegalBERT and Longformer","created":"2022-11-02","cited_by_count":20,"area":null},{"id":"2310.03281","title":"A 5' UTR Language Model for Decoding Untranslated Regions of mRNA and Function Predictions","created":"2023-10-05","cited_by_count":20,"area":null},{"id":"2406.19097","title":"Fairness and Bias in Multimodal AI: A Survey","created":"2024-06-27","cited_by_count":20,"area":"mllm"},{"id":"2112.13884","title":"A Fistful of Words: Learning Transferable Visual Models from Bag-of-Words Supervision","created":"2021-12-27","cited_by_count":19,"area":null},{"id":"2311.18805","title":"Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text","created":"2023-11-30","cited_by_count":19,"area":null},{"id":"2207.14393","title":"LAD: Language Models as Data for Zero-Shot Dialog","created":"2022-07-28","cited_by_count":18,"area":null},{"id":"2404.17806","title":"T-CLAP: Temporal-Enhanced Contrastive Language-Audio Pretraining","created":"2024-04-27","cited_by_count":18,"area":null},{"id":"2109.03777","title":"Bag-of-Words vs. Graph vs. Sequence in Text Classification: Questioning the Necessity of Text-Graphs and the Surprising Strength of a Wide MLP","created":"2021-09-08","cited_by_count":17,"area":null},{"id":"2109.12951","title":"Pragmatic competence of pre-trained language models through the lens of discourse connectives","created":"2021-09-27","cited_by_count":17,"area":null},{"id":"2203.08430","title":"Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure","created":"2022-03-16","cited_by_count":17,"area":null},{"id":"2210.12574","title":"The Curious Case of Absolute Position Embeddings","created":"2022-10-23","cited_by_count":17,"area":null},{"id":"2112.07605","title":"The King is Naked: on the Notion of Robustness for Natural Language Processing","created":"2021-12-13","cited_by_count":16,"area":null},{"id":"2203.10415","title":"How does the pre-training objective affect what large language models learn about linguistic properties?","created":"2022-03-20","cited_by_count":16,"area":null},{"id":"2405.03085","title":"Compressing Long Context for Enhancing RAG with AMR-based Concept Distillation","created":"2024-05-06","cited_by_count":16,"area":"rag"},{"id":"2104.07578","title":"Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models","created":"2021-04-15","cited_by_count":14,"area":null},{"id":"2109.02237","title":"BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks","created":"2021-09-06","cited_by_count":14,"area":null},{"id":"2208.04826","title":"Compositional Evaluation on Japanese Textual Entailment and Similarity","created":"2022-08-09","cited_by_count":14,"area":null},{"id":"2210.14328","title":"Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models","created":"2022-10-25","cited_by_count":14,"area":null},{"id":"2408.09639","title":"How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgments","created":"2024-08-19","cited_by_count":14,"area":null},{"id":"2302.14389","title":"Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context","created":"2023-02-28","cited_by_count":13,"area":null},{"id":"2204.10200","title":"An Exploratory Study on Code Attention in BERT","created":"2022-04-05","cited_by_count":12,"area":null},{"id":"2407.19638","title":"On the Reliability of Large Language Models for Causal Discovery","created":"2024-07-29","cited_by_count":12,"area":null},{"id":"2412.13989","title":"What makes a good metric? Evaluating automatic metrics for text-to-image consistency","created":"2024-12-18","cited_by_count":12,"area":null},{"id":"2209.03182","title":"On the Effectiveness of Compact Biomedical Transformers","created":"2022-09-07","cited_by_count":11,"area":"distill"},{"id":"2403.00758","title":"Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training","created":"2024-03-01","cited_by_count":11,"area":null},{"id":"2408.05506","title":"Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers","created":"2024-08-10","cited_by_count":11,"area":null},{"id":"2107.13955","title":"Local Structure Matters Most: Perturbation Study in NLU","created":"2021-07-29","cited_by_count":10,"area":null},{"id":"2109.14017","title":"Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations","created":"2021-09-28","cited_by_count":10,"area":null},{"id":"2205.03815","title":"Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence","created":"2022-05-08","cited_by_count":10,"area":null},{"id":"2309.16573","title":"Language Models as a Service: Overview of a New Paradigm and its Challenges","created":"2023-09-28","cited_by_count":10,"area":null},{"id":"2401.05300","title":"I am a Strange Dataset: Metalinguistic Tests for Language Models","created":"2024-01-10","cited_by_count":10,"area":null},{"id":"2109.04867","title":"Studying word order through iterative shuffling","created":"2021-09-10","cited_by_count":9,"area":null},{"id":"2109.08207","title":"Numerical reasoning in machine reading comprehension tasks: are we there yet?","created":"2021-09-16","cited_by_count":9,"area":null},{"id":"2202.12312","title":"Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies","created":"2022-02-24","cited_by_count":9,"area":null},{"id":"2401.15393","title":"Semantics of Multiword Expressions in Transformer-Based Models: A Survey","created":"2024-01-27","cited_by_count":9,"area":null},{"id":"2511.22470","title":"Hybrid, Unified and Iterative: A Novel Framework for Text-based Person Anomaly Retrieval","created":"2025-11-27","cited_by_count":9,"area":null},{"id":"2110.07240","title":"Causal Transformers Perform Below Chance on Recursive Nested Constructions, Unlike Humans","created":"2021-10-14","cited_by_count":8,"area":null},{"id":"2212.09864","title":"Synthetic Pre-Training Tasks for Neural Machine Translation","created":"2022-12-19","cited_by_count":8,"area":null},{"id":"2302.09865","title":"Can discrete information extraction prompts generalize across language models?","created":"2023-02-20","cited_by_count":8,"area":null},{"id":"2306.08889","title":"Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion","created":"2023-06-15","cited_by_count":8,"area":null},{"id":"2310.15829","title":"Unnatural language processing: How do language models handle machine-generated prompts?","created":"2023-10-24","cited_by_count":8,"area":null},{"id":"2310.17041","title":"On Surgical Fine-tuning for Language Encoders","created":"2023-10-25","cited_by_count":8,"area":null},{"id":"2503.01714","title":"Word Form Matters: LLMs' Semantic Reconstruction under Typoglycemia","created":"2025-03-03","cited_by_count":8,"area":null},{"id":"2605.19407","title":"A Bitter Lesson for Data Filtering","created":"2026-05-19","cited_by_count":8,"area":null},{"id":"2504.08001","title":"Linguistic Interpretability of Transformer-based Language Models: a systematic review","created":"2025-04-09","cited_by_count":7,"area":"mechinterp"},{"id":"2110.08887","title":"Schr\\\"odinger's Tree -- On Syntax and Neural Language Models","created":"2021-10-17","cited_by_count":6,"area":null},{"id":"2407.09184","title":"Does Incomplete Syntax Influence Korean Language Model? Focusing on Word Order and Case Markers","created":"2024-07-12","cited_by_count":6,"area":null},{"id":"2109.05388","title":"The Impact of Positional Encodings on Multilingual Compression","created":"2021-09-11","cited_by_count":5,"area":null},{"id":"2109.15101","title":"Compositional generalization in semantic parsing with pretrained transformers","created":"2021-09-30","cited_by_count":5,"area":null},{"id":"2201.12911","title":"Grammatical cues to subjecthood are redundant in a majority of simple clauses across languages","created":"2022-01-30","cited_by_count":5,"area":null},{"id":"2204.05695","title":"Self-Supervised Losses for One-Class Textual Anomaly Detection","created":"2022-04-12","cited_by_count":5,"area":null},{"id":"2210.06442","title":"Can Pretrained Language Models (Yet) Reason Deductively?","created":"2022-10-12","cited_by_count":5,"area":"lr"},{"id":"2311.07497","title":"Multilingual Nonce Dependency Treebanks: Understanding how Language Models represent and process syntactic structure","created":"2023-11-13","cited_by_count":5,"area":null},{"id":"2412.07754","title":"PortraitTalk: Towards Customizable One-Shot Audio-to-Talking Face Generation","created":"2024-12-10","cited_by_count":5,"area":null},{"id":"2506.15662","title":"CC-LEARN: Cohort-based Consistency Learning","created":"2025-06-18","cited_by_count":5,"area":null},{"id":"2106.03983","title":"Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference","created":"2021-06-07","cited_by_count":4,"area":null},{"id":"2110.05301","title":"On a Benefit of Mask Language Modeling: Robustness to Simplicity Bias","created":"2021-10-11","cited_by_count":4,"area":null}],"next_cursor":"100"}