{"list":"other","total":52,"rows":[{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"1612.00796","title":"Overcoming catastrophic forgetting in neural networks","created":"2016-12-02","cited_by_count":4787},{"id":"2202.05262","title":"Locating and Editing Factual Associations in GPT","created":"2022-02-10","cited_by_count":2691},{"id":"1802.07569","title":"Continual Lifelong Learning with Neural Networks: A Review","created":"2018-02-21","cited_by_count":1330},{"id":"2212.04089","title":"Editing Models with Task Arithmetic","created":"2022-12-08","cited_by_count":1120},{"id":"1511.05952","title":"Prioritized Experience Replay","created":"2015-11-18","cited_by_count":1099},{"id":"1811.11682","title":"Experience Replay for Continual Learning","created":"2018-11-28","cited_by_count":1095},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"2305.14233","title":"Enhancing Chat Language Models by Scaling High-quality Instructional Conversations","created":"2023-05-23","cited_by_count":839},{"id":"2109.01903","title":"Robust fine-tuning of zero-shot models","created":"2021-09-04","cited_by_count":797},{"id":"2202.10054","title":"Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution","created":"2022-02-21","cited_by_count":739},{"id":"2404.06395","title":"MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies","created":"2024-04-09","cited_by_count":723},{"id":"1903.08560","title":"Surprises in High-Dimensional Ridgeless Least Squares Interpolation","created":"2019-03-19","cited_by_count":513},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"2305.10429","title":"DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining","created":"2023-05-17","cited_by_count":399},{"id":"2005.00333","title":"XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning","created":"2020-05-01","cited_by_count":333},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"2102.01293","title":"Scaling Laws for Transfer","created":"2021-02-02","cited_by_count":272},{"id":"2309.14316","title":"Physics of Language Models: Part 3.1, Knowledge Storage and Extraction","created":"2023-09-25","cited_by_count":270},{"id":"2006.05987","title":"Revisiting Few-sample BERT Fine-tuning","created":"2020-06-10","cited_by_count":238},{"id":"2206.07137","title":"Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt","created":"2022-06-14","cited_by_count":236},{"id":"2405.05904","title":"Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?","created":"2024-05-09","cited_by_count":232},{"id":"1811.01088","title":"Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks","created":"2018-11-02","cited_by_count":215},{"id":"2402.05930","title":"WebLINX: Real-World Website Navigation with Multi-Turn Dialogue","created":"2024-02-08","cited_by_count":180},{"id":"2309.10105","title":"Understanding Catastrophic Forgetting in Language Models via Implicit Inference","created":"2023-09-18","cited_by_count":144},{"id":"2308.04014","title":"Continual Pre-Training of Large Language Models: How to (re)warm your model?","created":"2023-08-08","cited_by_count":137},{"id":"2403.08763","title":"Simple and Scalable Strategies to Continually Pre-train Large Language Models","created":"2024-03-13","cited_by_count":127},{"id":"2307.14430","title":"Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models","created":"2023-07-26","cited_by_count":121},{"id":"2310.17813","title":"A Spectral Condition for Feature Learning","created":"2023-10-26","cited_by_count":109},{"id":"2012.03107","title":"When Do Curricula Work?","created":"2020-12-05","cited_by_count":106},{"id":"2406.11813","title":"How Do Large Language Models Acquire Factual Knowledge During Pretraining?","created":"2024-06-17","cited_by_count":90},{"id":"2410.05192","title":"Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective","created":"2024-10-07","cited_by_count":81},{"id":"2404.07177","title":"Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic","created":"2024-04-10","cited_by_count":77},{"id":"2407.05872","title":"Scaling Exponents Across Parameterizations and Optimizers","created":"2024-07-08","cited_by_count":76},{"id":"2503.19206","title":"Overtrained Language Models Are Harder to Fine-Tune","created":"2025-03-24","cited_by_count":58},{"id":"2402.17400","title":"Investigating Continual Pretraining in Large Language Models: Insights and Implications","created":"2024-02-27","cited_by_count":57},{"id":"2402.12847","title":"Instruction-tuned Language Models are Better Knowledge Learners","created":"2024-02-20","cited_by_count":55},{"id":"2406.03476","title":"Does your data spark joy? Performance gains from domain upsampling at the end of training","created":"2024-06-05","cited_by_count":52},{"id":"2410.08527","title":"Scaling Laws for Predicting Downstream Performance in LLMs","created":"2024-10-11","cited_by_count":48},{"id":"2410.11820","title":"Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws","created":"2024-10-15","cited_by_count":47},{"id":"2409.05816","title":"Improving Pretraining Data Using Perplexity Correlations","created":"2024-09-09","cited_by_count":46},{"id":"2310.07831","title":"Optimal Linear Decay Learning Rate Schedules and Further Refinements","created":"2023-10-11","cited_by_count":43},{"id":"2403.20266","title":"Latxa: An Open Language Model and Evaluation Suite for Basque","created":"2024-03-29","cited_by_count":42},{"id":"2504.10478","title":"Weight Ensembling Improves Reasoning in Language Models","created":"2025-04-14","cited_by_count":38},{"id":"2501.18965","title":"The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training","created":"2025-01-31","cited_by_count":33},{"id":"2406.14785","title":"Understanding Finetuning for Factual Knowledge Extraction","created":"2024-06-20","cited_by_count":32},{"id":"2306.16921","title":"Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs","created":"2023-06-29","cited_by_count":22},{"id":"2505.17827","title":"Not All Tokens Are What You Need In Thinking","created":"2025-05-23","cited_by_count":20},{"id":"2310.15389","title":"Irreducible Curriculum for Language Model Pretraining","created":"2023-10-23","cited_by_count":16},{"id":"2410.05464","title":"Progressive distillation induces an implicit curriculum","created":"2024-10-07","cited_by_count":15},{"id":"2411.07175","title":"Continual Memorization of Factoids in Language Models","created":"2024-11-11","cited_by_count":8}],"next_cursor":null}