{"list":"other","total":78,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"1703.03400","title":"Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks","created":"2017-03-09","cited_by_count":4897},{"id":"2207.05221","title":"Language Models (Mostly) Know What They Know","created":"2022-07-11","cited_by_count":1856},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"2104.08786","title":"Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity","created":"2021-04-18","cited_by_count":1135},{"id":"2310.01798","title":"Large Language Models Cannot Self-Correct Reasoning Yet","created":"2023-10-03","cited_by_count":1017},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2302.00093","title":"Large Language Models Can Be Easily Distracted by Irrelevant Context","created":"2023-01-31","cited_by_count":841},{"id":"2305.11738","title":"CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing","created":"2023-05-19","cited_by_count":783},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2305.18654","title":"Faith and Fate: Limits of Transformers on Compositionality","created":"2023-05-29","cited_by_count":503},{"id":"2303.03846","title":"Larger language models do in-context learning differently","created":"2023-03-07","cited_by_count":394},{"id":"2303.17491","title":"Language Models can Solve Computer Tasks","created":"2023-03-30","cited_by_count":372},{"id":"1506.02216","title":"A Recurrent Latent Variable Model for Sequential Data","created":"2015-06-07","cited_by_count":362},{"id":"2210.01240","title":"Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought","created":"2022-10-03","cited_by_count":337},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2109.01247","title":"Do Prompt-Based Models Really Understand the Meaning of their Prompts?","created":"2021-09-02","cited_by_count":326},{"id":"2205.05055","title":"Data Distributional Properties Drive Emergent In-Context Learning in Transformers","created":"2022-04-22","cited_by_count":309},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2305.15408","title":"Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective","created":"2023-05-24","cited_by_count":294},{"id":"2305.12740","title":"Can We Edit Factual Knowledge by In-Context Learning?","created":"2023-05-22","cited_by_count":285},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2211.00053","title":"Generating Sequences by Learning to Self-Correct","created":"2022-10-31","cited_by_count":277},{"id":"1906.01820","title":"Risks from Learned Optimization in Advanced Machine Learning Systems","created":"2019-06-05","cited_by_count":258},{"id":"2307.02477","title":"Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks","created":"2023-07-05","cited_by_count":246},{"id":"2311.06668","title":"In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering","created":"2023-11-11","cited_by_count":245},{"id":"2110.11216","title":"User-friendly introduction to PAC-Bayes bounds","created":"2021-10-21","cited_by_count":239},{"id":"2308.03188","title":"Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies","created":"2023-08-06","cited_by_count":238},{"id":"2212.10001","title":"Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters","created":"2022-12-20","cited_by_count":222},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199},{"id":"2307.04721","title":"Large Language Models as General Pattern Machines","created":"2023-07-10","cited_by_count":197},{"id":"2106.06981","title":"Thinking Like Transformers","created":"2021-06-13","cited_by_count":184},{"id":"2309.13638","title":"Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve","created":"2023-09-24","cited_by_count":178},{"id":"2302.07459","title":"The Capacity for Moral Self-Correction in Large Language Models","created":"2023-02-15","cited_by_count":173},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161},{"id":"2305.09731","title":"What In-Context Learning \"Learns\" In-Context: Disentangling Task Recognition and Task Learning","created":"2023-05-16","cited_by_count":152},{"id":"2301.11916","title":"Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning","created":"2023-01-27","cited_by_count":149},{"id":"2306.15063","title":"Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression","created":"2023-06-26","cited_by_count":146},{"id":"2209.07686","title":"Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango","created":"2022-09-16","cited_by_count":128},{"id":"2303.07895","title":"The Learnability of In-Context Learning","created":"2023-03-14","cited_by_count":127},{"id":"2305.19420","title":"What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization","created":"2023-05-30","cited_by_count":118},{"id":"2312.03002","title":"The mechanistic basis of data dependence and abrupt learning in an in-context classification task","created":"2023-12-03","cited_by_count":118},{"id":"2205.12685","title":"Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations","created":"2022-05-25","cited_by_count":116},{"id":"2307.15936","title":"A Theory for Emergence of Complex Skills in Language Models","created":"2023-07-29","cited_by_count":113},{"id":"2310.17191","title":"How do Language Models Bind Entities in Context?","created":"2023-10-26","cited_by_count":108},{"id":"2105.11115","title":"Self-Attention Networks Can Process Bounded Hierarchical Languages","created":"2021-05-24","cited_by_count":105},{"id":"2304.03843","title":"Why think step by step? Reasoning emerges from the locality of experience","created":"2023-04-07","cited_by_count":103},{"id":"2309.05858","title":"Uncovering mesa-optimization algorithms in Transformers","created":"2023-09-11","cited_by_count":100},{"id":"2310.01082","title":"Linear attention is (maybe) all you need (to understand transformer optimization)","created":"2023-10-02","cited_by_count":98},{"id":"2310.08391","title":"How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?","created":"2023-10-12","cited_by_count":96},{"id":"2303.07971","title":"A Theory of Emergent In-Context Learning as Implicit Structure Induction","created":"2023-03-14","cited_by_count":86},{"id":"2306.04891","title":"In-Context Learning through the Bayesian Prism","created":"2023-06-08","cited_by_count":83},{"id":"2311.08360","title":"The Transient Nature of Emergent In-Context Learning in Transformers","created":"2023-11-14","cited_by_count":76},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2211.13892","title":"Complementary Explanations for Effective In-Context Learning","created":"2022-11-25","cited_by_count":71},{"id":"2305.17256","title":"Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning","created":"2023-05-26","cited_by_count":69},{"id":"2312.06528","title":"Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context","created":"2023-12-11","cited_by_count":63},{"id":"2301.09790","title":"The Next Chapter: A Study of Large Language Models in Storytelling","created":"2023-01-24","cited_by_count":58},{"id":"2210.05675","title":"Transformers generalize differently from information stored in context vs in weights","created":"2022-10-11","cited_by_count":49},{"id":"2202.05798","title":"The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention","created":"2022-02-11","cited_by_count":46},{"id":"2311.00871","title":"Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models","created":"2023-11-01","cited_by_count":45},{"id":"2307.01163","title":"Improving Language Plasticity via Pretraining with Active Forgetting","created":"2023-07-03","cited_by_count":43},{"id":"2304.13276","title":"The Closeness of In-Context Learning and Weight Shifting for Softmax Regression","created":"2023-04-26","cited_by_count":41},{"id":"2311.07954","title":"A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning","created":"2023-11-14","cited_by_count":41},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40},{"id":"2401.15530","title":"An Information-Theoretic Analysis of In-Context Learning","created":"2024-01-28","cited_by_count":40},{"id":"2310.20046","title":"Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection","created":"2023-10-30","cited_by_count":37},{"id":"2311.08993","title":"When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks","created":"2023-11-15","cited_by_count":37},{"id":"2011.05268","title":"Towards Interpretable Natural Language Understanding with Explanations as Latent Variables","created":"2020-10-24","cited_by_count":30},{"id":"1908.11527","title":"Implicit Deep Latent Variable Models for Text Generation","created":"2019-08-30","cited_by_count":26},{"id":"2305.16704","title":"A Closer Look at In-Context Learning under Distribution Shifts","created":"2023-05-26","cited_by_count":25},{"id":"2307.01201","title":"Schema-learning and rebinding as mechanisms of in-context learning and emergence","created":"2023-06-16","cited_by_count":14},{"id":"2311.03498","title":"In-Context Exemplars as Clues to Retrieving from Large Associative Memory","created":"2023-11-06","cited_by_count":13},{"id":"1812.06834","title":"A Tutorial on Deep Latent Variable Models of Natural Language","created":"2018-12-17","cited_by_count":10},{"id":"1905.00507","title":"Learning higher-order sequential structure with cloned HMMs","created":"2019-05-01","cited_by_count":10}],"next_cursor":null}