# Task Transfer by Preference-Based Cost Learning

arXiv: https://arxiv.org/abs/1805.04686 · HTML: https://mlsanity.com/paper/1805.04686/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2018.

## Built on by (most cited first, 18 of 18)

- [Latent Diffusion Energy-Based Model for Interpretable Text Modeling](https://mlsanity.com/paper/2206.05895/) (2022)
- [Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement](https://mlsanity.com/paper/1910.04417/) (2019)
- [Human irrationality: both bad and good for reward inference](https://mlsanity.com/paper/2111.06956/) (2021)
- [Searching for Fast Model Families on Datacenter Accelerators](https://mlsanity.com/paper/2102.05610/) (2021)
- [ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs](https://mlsanity.com/paper/2411.13547/) (2024)
- [Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitation](https://mlsanity.com/paper/2411.09891/) (2024)
- [FLASH: Fast Neural Architecture Search with Hardware Optimization](https://mlsanity.com/paper/2108.00568/) (2021)
- [FinGAIA: A Chinese Benchmark for AI Agents in Real-World Financial Domain](https://mlsanity.com/paper/2507.17186/) (2025)
- [OutlierNets: Highly Compact Deep Autoencoder Network Architectures for On-Device Acoustic Anomaly Detection](https://mlsanity.com/paper/2104.00528/) (2021)
- [Measuring what Really Matters: Optimizing Neural Networks for TinyML](https://mlsanity.com/paper/2104.10645/) (2021)
- [Lightweight Monocular Depth with a Novel Neural Architecture Search Method](https://mlsanity.com/paper/2108.11105/) (2021)
- [MAPLE: Microprocessor A Priori for Latency Estimation](https://mlsanity.com/paper/2111.15106/) (2021)
- [AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge](https://mlsanity.com/paper/2104.14623/) (2021)
- [COVID-Net US: A Tailored, Highly Efficient, Self-Attention Deep Convolutional Neural Network Design for Detection of COVID-19 Patient Cases from Point-of-care Ultrasound Imaging](https://mlsanity.com/paper/2108.03131/) (2021)
- [Network Space Search for Pareto-Efficient Spaces](https://mlsanity.com/paper/2104.11014/) (2021)
- [Joint Channel and Weight Pruning for Model Acceleration on Moblie Devices](https://mlsanity.com/paper/2110.08013/) (2021)
- [Fast Neural Architecture Search for Lightweight Dense Prediction Networks](https://mlsanity.com/paper/2203.01994/) (2022)
- [ISTRBoost: Importance Sampling Transfer Regression using Boosting](https://mlsanity.com/paper/2204.12044/) (2022)

## Abstract (arXiv)

The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task. Given their successes on robotic action planning, current methods mostly rely on two requirements: exactly-relevant expert demonstrations or the explicitly-coded cost function on target task, both of which, however, are inconvenient to obtain in practice. In this paper, we relax these two strong conditions by developing a novel task transfer framework where the expert preference is applied as a guidance. In particular, we alternate the following two steps: Firstly, letting experts apply pre-defined preference rules to select related expert demonstrates for the target task. Secondly, based on the selection result, we learn the target cost function and trajectory distribution simultaneously via enhanced Adversarial MaxEnt IRL and generate more trajectories by the learned target distribution for the next preference selection. The theoretical analysis on the distribution learning and convergence of the proposed algorithm are provided. Extensive simulations on several benchmarks have been conducted for further verifying the effectiveness of the proposed method.