# DFF: Decision-Focused Fine-tuning for Smarter Predict-then-Optimize with Limited Data

arXiv: https://arxiv.org/abs/2501.01874 · HTML: https://mlsanity.com/paper/2501.01874/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 3 Jan 2025.

## Builds on (most cited first, 24 of 24)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention](https://mlsanity.com/paper/2303.16199/) (2023)
- [OptNet: Differentiable Optimization as a Layer in Neural Networks](https://mlsanity.com/paper/1703.00443/) (2017)
- [Smart "Predict, then Optimize"](https://mlsanity.com/paper/1710.08005/) (2017)
- [Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization](https://mlsanity.com/paper/1809.05504/) (2018)
- [Model-Ensemble Trust-Region Policy Optimization](https://mlsanity.com/paper/1802.10592/) (2018)
- [End-to-End Constrained Optimization Learning: A Survey](https://mlsanity.com/paper/2103.16378/) (2021)
- [Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation](https://mlsanity.com/paper/1708.05144/) (2017)
- [Smart Predict-and-Optimize for Hard Combinatorial Optimization Problems](https://mlsanity.com/paper/1911.10092/) (2019)
- [Interior Point Solving for LP-based prediction+optimisation](https://mlsanity.com/paper/2010.13943/) (2020)
- [Decision-Focused Learning: Through the Lens of Learning to Rank](https://mlsanity.com/paper/2112.03609/) (2021)
- [Decision Trees for Decision-Making under the Predict-then-Optimize Framework](https://mlsanity.com/paper/2003.00360/) (2020)
- [Fast Rates for Contextual Linear Optimization](https://mlsanity.com/paper/2011.03030/) (2020)
- [A Unified Framework for Marketing Budget Allocation](https://mlsanity.com/paper/1902.01128/) (2019)
- [Learning with Combinatorial Optimization Layers: a Probabilistic Approach](https://mlsanity.com/paper/2207.13513/) (2022)
- [End-to-end Learning for Fair Ranking Systems](https://mlsanity.com/paper/2111.10723/) (2021)
- [Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize](https://mlsanity.com/paper/2305.16830/) (2023)
- [Predict+Optimize for Packing and Covering LPs with Unknown Parameters in Constraints](https://mlsanity.com/paper/2209.03668/) (2022)
- [TaskMet: Task-Driven Metric Learning for Model Learning](https://mlsanity.com/paper/2312.05250/) (2023)
- [Uncertainty-Aware Policy Optimization: A Robust, Adaptive Trust Region Approach](https://mlsanity.com/paper/2012.10791/) (2020)
- [A Framework for Multi-stage Bonus Allocation in meal delivery Platform](https://mlsanity.com/paper/2202.10695/) (2022)
- [Gradient boosting for convex cone predict and optimize problems](https://mlsanity.com/paper/2204.06895/) (2022)
- [PILA: Physics-Informed Low Rank Augmentation for Interpretable Earth Observation](https://mlsanity.com/paper/2405.18953/) (2024)

## Built on by (most cited first, 1 of 1)

- [Decision-Focused Continual Learning for Seaport Power-Logistics Scheduling: Generalization across Varying Tasks](https://mlsanity.com/paper/2511.07938/) (2025)

## Abstract (arXiv)

Decision-focused learning (DFL) offers an end-to-end approach to the predict-then-optimize (PO) framework by training predictive models directly on decision loss (DL), enhancing decision-making performance within PO contexts. However, the implementation of DFL poses distinct challenges. Primarily, DL can result in deviation from the physical significance of the predictions under limited data. Additionally, some predictive models are non-differentiable or black-box, which cannot be adjusted using gradient-based methods. To tackle the above challenges, we propose a novel framework, Decision-Focused Fine-tuning (DFF), which embeds the DFL module into the PO pipeline via a novel bias correction module. DFF is formulated as a constrained optimization problem that maintains the proximity of the DL-enhanced model to the original predictive model within a defined trust region. We theoretically prove that DFF strictly confines prediction bias within a predetermined upper bound, even with limited datasets, thereby substantially reducing prediction shifts caused by DL under limited data. Furthermore, the bias correction module can be integrated into diverse predictive models, enhancing adaptability to a broad range of PO tasks. Extensive evaluations on synthetic and real-world datasets, including network flow, portfolio optimization, and resource allocation problems with different predictive models, demonstrate that DFF not only improves decision performance but also adheres to fine-tuning constraints, showcasing robust adaptability across various scenarios.