{"list":"other","total":79,"rows":[{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2305.18290","title":"Direct Preference Optimization: Your Language Model is Secretly a Reward Model","created":"2023-05-29","cited_by_count":8065},{"id":"1706.03741","title":"Deep reinforcement learning from human preferences","created":"2017-06-12","cited_by_count":3912},{"id":"2204.05862","title":"Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback","created":"2022-04-12","cited_by_count":3720},{"id":"1909.08593","title":"Fine-Tuning Language Models from Human Preferences","created":"2019-09-18","cited_by_count":2150},{"id":"2112.09332","title":"WebGPT: Browser-assisted question-answering with human feedback","created":"2021-12-17","cited_by_count":1745},{"id":"2310.13548","title":"Towards Understanding Sycophancy in Language Models","created":"2023-10-20","cited_by_count":1040},{"id":"2405.14734","title":"SimPO: Simple Preference Optimization with a Reference-Free Reward","created":"2024-05-23","cited_by_count":963},{"id":"2310.12036","title":"A General Theoretical Paradigm to Understand Learning from Human Preferences","created":"2023-10-18","cited_by_count":912},{"id":"2302.04023","title":"A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity","created":"2023-02-08","cited_by_count":894},{"id":"2307.15217","title":"Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback","created":"2023-07-27","cited_by_count":759},{"id":"2202.10054","title":"Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution","created":"2022-02-21","cited_by_count":739},{"id":"2311.03658","title":"The Linear Representation Hypothesis and the Geometry of Large Language Models","created":"2023-11-07","cited_by_count":716},{"id":"2304.06767","title":"RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment","created":"2023-04-13","cited_by_count":686},{"id":"2012.13255","title":"Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning","created":"2020-12-22","cited_by_count":671},{"id":"2310.12773","title":"Safe RLHF: Safe Reinforcement Learning from Human Feedback","created":"2023-10-19","cited_by_count":656},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"1509.01240","title":"Train faster, generalize better: Stability of stochastic gradient descent","created":"2015-09-03","cited_by_count":552},{"id":"1703.11008","title":"Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data","created":"2017-03-31","cited_by_count":533},{"id":"2209.14375","title":"Improving alignment of dialogue agents via targeted human judgements","created":"2022-09-28","cited_by_count":531},{"id":"2304.05302","title":"RRHF: Rank Responses to Align Language Models with Human Feedback without tears","created":"2023-04-11","cited_by_count":463},{"id":"1811.07871","title":"Scalable agent alignment via reward modeling: a research direction","created":"2018-11-19","cited_by_count":447},{"id":"2306.16388","title":"Towards Measuring the Representation of Subjective Global Opinions in Language Models","created":"2023-06-28","cited_by_count":437},{"id":"2312.11456","title":"Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint","created":"2023-12-18","cited_by_count":350},{"id":"1811.04918","title":"Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers","created":"2018-11-12","cited_by_count":340},{"id":"2201.03544","title":"The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models","created":"2022-01-10","cited_by_count":328},{"id":"2309.06657","title":"Statistical Rejection Sampling Improves Preference Optimization","created":"2023-09-13","cited_by_count":323},{"id":"1802.05296","title":"Stronger generalization bounds for deep nets via a compression approach","created":"2018-02-14","cited_by_count":307},{"id":"2106.05091","title":"PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training","created":"2021-06-09","cited_by_count":302},{"id":"2306.17492","title":"Preference Ranking Optimization for Human Alignment","created":"2023-06-30","cited_by_count":285},{"id":"2402.01306","title":"KTO: Model Alignment as Prospect Theoretic Optimization","created":"2024-02-02","cited_by_count":264},{"id":"2310.19852","title":"AI Alignment: A Comprehensive Survey","created":"2023-10-30","cited_by_count":263},{"id":"2404.10719","title":"Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study","created":"2024-04-16","cited_by_count":262},{"id":"1906.01820","title":"Risks from Learned Optimization in Advanced Machine Learning Systems","created":"2019-06-05","cited_by_count":258},{"id":"2209.00626","title":"The Alignment Problem from a Deep Learning Perspective","created":"2022-08-30","cited_by_count":258},{"id":"2301.11270","title":"Principled Reinforcement Learning with Human Feedback from Pairwise or $K$-wise Comparisons","created":"2023-01-26","cited_by_count":247},{"id":"2404.12358","title":"From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function","created":"2024-04-18","cited_by_count":242},{"id":"2402.13228","title":"Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive","created":"2024-02-20","cited_by_count":241},{"id":"2312.00886","title":"Nash Learning from Human Feedback","created":"2023-12-01","cited_by_count":224},{"id":"2308.14752","title":"AI Deception: A Survey of Examples, Risks, and Potential Solutions","created":"2023-08-28","cited_by_count":188},{"id":"2010.15110","title":"Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel","created":"2020-10-28","cited_by_count":187},{"id":"2404.14367","title":"Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data","created":"2024-04-22","cited_by_count":185},{"id":"2305.15324","title":"Model evaluation for extreme risks","created":"2023-05-24","cited_by_count":177},{"id":"2402.10207","title":"Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustment","created":"2024-02-15","cited_by_count":175},{"id":"2402.05749","title":"Generalized Preference Optimization: A Unified Approach to Offline Alignment","created":"2024-02-08","cited_by_count":157},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150},{"id":"2210.11369","title":"On Feature Learning in the Presence of Spurious Correlations","created":"2022-10-20","cited_by_count":145},{"id":"2309.00667","title":"Taken out of context: On measuring situational awareness in LLMs","created":"2023-09-01","cited_by_count":143},{"id":"2206.11871","title":"Offline RL for Natural Language Generation with Implicit Language Q Learning","created":"2022-06-05","cited_by_count":138},{"id":"2403.00409","title":"Provably Robust DPO: Aligning Language Models with Noisy Feedback","created":"2024-03-01","cited_by_count":135},{"id":"2404.11999","title":"Token-level Direct Preference Optimization","created":"2024-04-18","cited_by_count":135},{"id":"2210.01790","title":"Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals","created":"2022-10-04","cited_by_count":126},{"id":"2402.08925","title":"MaxMin-RLHF: Alignment with Diverse Human Preferences","created":"2024-02-14","cited_by_count":123},{"id":"2210.00045","title":"Calibrating Sequence likelihood Improves Conditional Language Generation","created":"2022-09-30","cited_by_count":121},{"id":"2002.10826","title":"Deep Representation Learning on Long-tailed Data: A Learnable Embedding Augmentation Perspective","created":"2020-02-25","cited_by_count":120},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"2403.08635","title":"Human Alignment of Large Language Models through Online Preference Optimisation","created":"2024-03-13","cited_by_count":91},{"id":"2205.11140","title":"Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation","created":"2022-05-23","cited_by_count":81},{"id":"2310.13639","title":"Contrastive Preference Learning: Learning from Human Feedback without RL","created":"2023-10-20","cited_by_count":79},{"id":"2403.03867","title":"On the Origins of Linear Representations in Large Language Models","created":"2024-03-06","cited_by_count":79},{"id":"2211.13609","title":"PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization","created":"2022-11-24","cited_by_count":76},{"id":"2402.02992","title":"Decoding-time Realignment of Language Models","created":"2024-02-05","cited_by_count":67},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63},{"id":"1902.01384","title":"Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks","created":"2019-02-04","cited_by_count":60},{"id":"2402.08114","title":"Active Preference Learning for Large Language Models","created":"2024-02-12","cited_by_count":58},{"id":"2406.08469","title":"PAL: Pluralistic Alignment Framework for Learning from Heterogeneous Preferences","created":"2024-06-12","cited_by_count":54},{"id":"2312.17173","title":"Non-Vacuous Generalization Bounds for Large Language Models","created":"2023-12-28","cited_by_count":53},{"id":"2405.19534","title":"Preference Learning Algorithms Do Not Learn Preference Rankings","created":"2024-05-29","cited_by_count":48},{"id":"2303.09387","title":"Characterizing Manipulation from AI Systems","created":"2023-03-16","cited_by_count":43},{"id":"1702.08712","title":"Algorithmic stability and hypothesis complexity","created":"2017-02-28","cited_by_count":39},{"id":"2012.04115","title":"Generalization bounds for deep learning","created":"2020-12-07","cited_by_count":38},{"id":"2403.01857","title":"Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences","created":"2024-03-04","cited_by_count":27},{"id":"2401.11458","title":"Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback","created":"2024-01-21","cited_by_count":25},{"id":"2403.18742","title":"Understanding the Learning Dynamics of Alignment with Human Feedback","created":"2024-03-27","cited_by_count":15},{"id":"2206.01399","title":"Generalization for multiclass classification with overparameterized linear models","created":"2022-06-03","cited_by_count":8},{"id":"2405.16964","title":"Exploring the LLM Journey from Cognition to Expression with Linear Representations","created":"2024-05-27","cited_by_count":5}],"next_cursor":null}