{"list":"closest","total":122,"rows":[{"id":"2502.14340","title":"Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective","created":"2025-02-20","cited_by_count":11},{"id":"2502.13922","title":"LongPO: Long Context Self-Evolution of Large Language Models through Short-to-Long Preference Optimization","created":"2025-02-19","cited_by_count":24},{"id":"2502.05400","title":"Dynamic Noise Preference Optimization: Self-Improvement of Large Language Models with Self-Synthetic Data","created":"2025-02-08","cited_by_count":4},{"id":"2502.00883","title":"SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters","created":"2025-02-02","cited_by_count":25},{"id":"2501.12735","title":"Online Preference Alignment for Language Models via Count-based Exploration","created":"2025-01-22","cited_by_count":26},{"id":"2501.03271","title":"DPO Kernels: A Semantically-Aware, Kernel-Enhanced, and Divergence-Rich Paradigm for Direct Preference Optimization","created":"2025-01-05","cited_by_count":2},{"id":"2501.01821","title":"SDPO: Segment-Level Direct Preference Optimization for Social Agents","created":"2025-01-03","cited_by_count":30},{"id":"2412.20299","title":"No Preference Left Behind: Group Distributional Preference Optimization","created":"2024-12-28","cited_by_count":21},{"id":"2412.17696","title":"Understanding the Logic of Direct Preference Alignment through Logic","created":"2024-12-23","cited_by_count":5},{"id":"2412.14516","title":"Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment","created":"2024-12-19","cited_by_count":51},{"id":"2411.07618","title":"Constrain Alignment with Sparse Autoencoders","created":"2024-11-12","cited_by_count":12},{"id":"2411.06208","title":"IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization","created":"2024-11-09","cited_by_count":31},{"id":"2411.02442","title":"TODO: Enhancing LLM Alignment with Ternary Preferences","created":"2024-11-02","cited_by_count":5},{"id":"2410.18585","title":"Aligning CodeLLMs with Direct Preference Optimization","created":"2024-10-24","cited_by_count":9},{"id":"2410.15595","title":"A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applications","created":"2024-10-21","cited_by_count":27},{"id":"2410.18127","title":"Holistic Utility Preference Learning for Listwise Alignment","created":"2024-10-17","cited_by_count":2},{"id":"2410.13828","title":"A Common Pitfall of Margin-based Language Model Alignment: Gradient Entanglement","created":"2024-10-17","cited_by_count":10},{"id":"2410.12138","title":"Preference Optimization with Multi-Sample Comparisons","created":"2024-10-16","cited_by_count":16},{"id":"2410.11677","title":"Understanding Likelihood Over-optimisation in Direct Alignment Algorithms","created":"2024-10-15","cited_by_count":7},{"id":"2410.10148","title":"AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization","created":"2024-10-14","cited_by_count":29},{"id":"2410.09362","title":"SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Margins","created":"2024-10-12","cited_by_count":5},{"id":"2410.08847","title":"Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization","created":"2024-10-11","cited_by_count":67},{"id":"2410.08458","title":"Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both","created":"2024-10-11","cited_by_count":5},{"id":"2410.07163","title":"Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning","created":"2024-10-09","cited_by_count":172},{"id":"2410.06293","title":"Accelerated Preference Optimization for Large Language Model Alignment","created":"2024-10-08","cited_by_count":5},{"id":"2410.05102","title":"SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masks","created":"2024-10-07","cited_by_count":9},{"id":"2410.04350","title":"TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights","created":"2024-10-06","cited_by_count":42},{"id":"2410.04346","title":"Permutative Preference Alignment from Listwise Ranking of Human Judgments","created":"2024-10-06","cited_by_count":7},{"id":"2410.04203","title":"RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization","created":"2024-10-05","cited_by_count":19},{"id":"2410.00508","title":"FlipGuard: Defending Preference Alignment against Update Regression with Constrained Optimization","created":"2024-10-01","cited_by_count":2},{"id":"2409.19605","title":"The Crucial Role of Samplers in Online Direct Preference Optimization","created":"2024-09-29","cited_by_count":27},{"id":"2409.17431","title":"On Extending Direct Preference Optimization to Accommodate Ties","created":"2024-09-25","cited_by_count":8},{"id":"2409.11564","title":"Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey","created":"2024-09-17","cited_by_count":23},{"id":"2409.08845","title":"AIPO: Improving Training Objective for Iterative Preference Optimization","created":"2024-09-13","cited_by_count":5},{"id":"2409.06691","title":"Geometric-Averaged Preference Optimization for Soft Preference Labels","created":"2024-09-10","cited_by_count":18},{"id":"2409.03650","title":"On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization","created":"2024-09-05","cited_by_count":27},{"id":"2409.02795","title":"Towards a Unified View of Preference Learning for Large Language Models: A Survey","created":"2024-09-04","cited_by_count":17},{"id":"2408.13518","title":"Selective Preference Optimization via Token-Level Reward Function Estimation","created":"2024-08-24","cited_by_count":25},{"id":"2408.07471","title":"Bridging and Modeling Correlations in Pairwise Data for Direct Preference Optimization","created":"2024-08-14","cited_by_count":7},{"id":"2408.06266","title":"Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment","created":"2024-08-12","cited_by_count":46},{"id":"2407.18676","title":"Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift","created":"2024-07-26","cited_by_count":13},{"id":"2407.14477","title":"Data-Centric Human Preference with Rationales for Direct Preference Alignment","created":"2024-07-19","cited_by_count":5},{"id":"2407.13709","title":"Understanding Reference Policies in Direct Preference Optimization","created":"2024-07-18","cited_by_count":22},{"id":"2407.13399","title":"Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization","created":"2024-07-18","cited_by_count":78},{"id":"2407.07880","title":"Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization","created":"2024-07-10","cited_by_count":53},{"id":"2407.03051","title":"Improving Conversational Abilities of Quantized Large Language Models via Direct Preference Alignment","created":"2024-07-03","cited_by_count":9},{"id":"2406.17923","title":"PAFT: A Parallel Training Paradigm for Effective LLM Fine-Tuning","created":"2024-06-25","cited_by_count":11},{"id":"2406.14868","title":"Direct Multi-Turn Preference Optimization for Language Agents","created":"2024-06-21","cited_by_count":64},{"id":"2406.10957","title":"Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence","created":"2024-06-16","cited_by_count":31},{"id":"2406.09279","title":"Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference Feedback","created":"2024-06-13","cited_by_count":107},{"id":"2406.08414","title":"Discovering Preference Optimization Algorithms with and for Large Language Models","created":"2024-06-12","cited_by_count":42},{"id":"2406.07657","title":"OPTune: Efficient Online Preference Tuning","created":"2024-06-11","cited_by_count":10},{"id":"2406.07327","title":"3D-Properties: Identifying Challenges in DPO and Charting a Path Forward","created":"2024-06-11","cited_by_count":24},{"id":"2406.05882","title":"Distributional Preference Alignment of LLMs via Optimal Transport","created":"2024-06-09","cited_by_count":26},{"id":"2406.05534","title":"Online DPO: Online Direct Preference Optimization with Fast-Slow Chasing","created":"2024-06-08","cited_by_count":39},{"id":"2406.04879","title":"A Deep Dive into the Trade-Offs of Parameter-Efficient Preference Alignment Techniques","created":"2024-06-07","cited_by_count":16},{"id":"2406.04412","title":"Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment","created":"2024-06-06","cited_by_count":27},{"id":"2406.01771","title":"LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback","created":"2024-06-03","cited_by_count":51},{"id":"2406.01660","title":"Self-Improving Robust Preference Optimization","created":"2024-06-03","cited_by_count":19},{"id":"2406.01462","title":"The Importance of Online Data: Understanding Preference Fine-tuning via Coverage","created":"2024-06-03","cited_by_count":55},{"id":"2406.00832","title":"BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Sampling","created":"2024-06-02","cited_by_count":121},{"id":"2405.21040","title":"Direct Alignment of Language Models via Quality-Aware Self-Refinement","created":"2024-05-31","cited_by_count":7},{"id":"2405.20830","title":"Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment","created":"2024-05-31","cited_by_count":7},{"id":"2405.19534","title":"Preference Learning Algorithms Do Not Learn Preference Rankings","created":"2024-05-29","cited_by_count":48},{"id":"2405.19332","title":"Self-Exploring Language Models: Active Preference Elicitation for Online Alignment","created":"2024-05-29","cited_by_count":63},{"id":"2405.19316","title":"Robust Preference Optimization through Reward Model Distillation","created":"2024-05-29","cited_by_count":67},{"id":"2405.19107","title":"Offline Regularised Reinforcement Learning for Large Language Models Alignment","created":"2024-05-29","cited_by_count":48},{"id":"2405.17956","title":"Unified Preference Optimization: Language Model Alignment Beyond the Preference Frontier","created":"2024-05-28","cited_by_count":4},{"id":"2405.17931","title":"Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment","created":"2024-05-28","cited_by_count":28},{"id":"2405.15065","title":"Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferences","created":"2024-05-23","cited_by_count":23},{"id":"2405.14953","title":"MallowsPO: Fine-Tune Your LLM with Preference Dispersions","created":"2024-05-23","cited_by_count":18},{"id":"2405.14734","title":"SimPO: Simple Preference Optimization with a Reference-Free Reward","created":"2024-05-23","cited_by_count":963},{"id":"2405.13516","title":"LIRE: listwise reward enhancement for preference alignment","created":"2024-05-22","cited_by_count":11},{"id":"2405.12739","title":"SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling","created":"2024-05-21","cited_by_count":21},{"id":"2405.09223","title":"Word Alignment as Preference for Machine Translation","created":"2024-05-15","cited_by_count":14},{"id":"2405.08448","title":"Understanding the performance gap between online and offline alignment algorithms","created":"2024-05-14","cited_by_count":110},{"id":"2405.01511","title":"D2PO: Discriminator-Guided DPO with Response Evaluation Models","created":"2024-05-02","cited_by_count":9},{"id":"2405.00747","title":"Soft Preference Optimization: Aligning Language Models to Expert Distributions","created":"2024-04-30","cited_by_count":9},{"id":"2404.14723","title":"Insights into Alignment: Evaluating DPO and its Variants Across Multiple Tasks","created":"2024-04-23","cited_by_count":35},{"id":"2404.13846","title":"Filtered Direct Preference Optimization","created":"2024-04-22","cited_by_count":24},{"id":"2404.12358","title":"From $r$ to $Q^*$: Your Language Model is Secretly a Q-Function","created":"2024-04-18","cited_by_count":242},{"id":"2404.11999","title":"Token-level Direct Preference Optimization","created":"2024-04-18","cited_by_count":135},{"id":"2404.11049","title":"Stepwise Alignment for Constrained Language Model Policy Optimization","created":"2024-04-17","cited_by_count":30},{"id":"2404.10719","title":"Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study","created":"2024-04-16","cited_by_count":262},{"id":"2404.09824","title":"Impact of Preference Noise on the Alignment Performance of Generative Language Models","created":"2024-04-15","cited_by_count":41},{"id":"2404.09656","title":"Learn Your Reference Model for Real Good Alignment","created":"2024-04-15","cited_by_count":57},{"id":"2404.05868","title":"Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning","created":"2024-04-08","cited_by_count":504},{"id":"2404.04656","title":"Binary Classifier Optimization for Large Language Model Alignment","created":"2024-04-06","cited_by_count":45},{"id":"2404.04626","title":"Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective","created":"2024-04-06","cited_by_count":69},{"id":"2403.19443","title":"Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model","created":"2024-03-28","cited_by_count":25},{"id":"2403.19270","title":"sDPO: Don't Use Your Data All at Once","created":"2024-03-28","cited_by_count":46},{"id":"2403.19159","title":"Disentangling Length from Quality in Direct Preference Optimization","created":"2024-03-28","cited_by_count":196},{"id":"2403.08635","title":"Human Alignment of Large Language Models through Online Preference Optimisation","created":"2024-03-13","cited_by_count":91},{"id":"2403.07691","title":"ORPO: Monolithic Preference Optimization without Reference Model","created":"2024-03-12","cited_by_count":562},{"id":"2403.07230","title":"Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences","created":"2024-03-12","cited_by_count":39},{"id":"2403.03419","title":"Negating Negatives: Alignment with Human Negative Samples via Distributional Dispreference Optimization","created":"2024-03-06","cited_by_count":11},{"id":"2403.00409","title":"Provably Robust DPO: Aligning Language Models with Noisy Feedback","created":"2024-03-01","cited_by_count":135},{"id":"2402.19085","title":"Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment","created":"2024-02-29","cited_by_count":107},{"id":"2402.13228","title":"Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive","created":"2024-02-20","cited_by_count":241},{"id":"2402.11907","title":"Direct Large Language Model Alignment Through Self-Rewarding Contrastive Prompt Distillation","created":"2024-02-19","cited_by_count":18}],"next_cursor":"100"}