mlsanityData to 7 Oct 2026

Auxiliary Learning by Implicit Differentiation

arXiv 2007.02693 ↗First posted 22 Jun 2020cs.CV cs.LG stat.MLcited by 45

Not on the map. Why?

Abstract (arXiv)

Training neural networks with auxiliary tasks is a common practice for improving the performance on a main task of interest. Two main challenges arise in this multi-task learning setting: (i) designing useful auxiliary tasks; and (ii) combining auxiliary tasks into a single coherent loss. Here, we propose a novel framework, AuxiLearn, that targets both challenges based on implicit differentiation. First, when useful auxiliaries are known, we propose learning a network that combines all losses into a single coherent objective function. This network can learn non-linear interactions between tasks. Second, when no useful auxiliary task is known, we describe how to learn a network that generates a meaningful, novel auxiliary task. We evaluate AuxiLearn in a series of tasks and domains, including image segmentation and learning with attributes in the low data regime, and find that it consistently outperforms competing methods.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 10; 2023: 9; 2024: 9; 2025: 9; 2026: 3.

45 papers.

Sort:

By area:

  1. 2022Multi-Task Learning as a Bargaining Gamecited by 209
  2. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  3. 2022AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learningcited by 31
  4. 2023ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learningcited by 28
  5. 2023Prismer: A Vision-Language Model with Multi-Task Expertscited by 28
  6. 2021Meta-Learning to Improve Pre-Trainingcited by 27
  7. 2021Should We Be Pre-training? An Argument for End-task Aware Training as an Alternativecited by 26
  8. 2021Pareto Domain Adaptationcited by 20
  9. 2024Functional Bilevel Optimization for Machine Learningcited by 20
  10. 2023MELTR: Meta Loss Transformer for Learning to Fine-tune Video Foundation Modelscited by 15
  11. 2023Improving Few-Shot Generalization by Exploring and Exploiting Auxiliary Datacited by 13
  12. 2022AANG: Automating Auxiliary Learningcited by 12
33 more
  1. 2022Optimization with Access to Auxiliary Informationcited by 12
  2. 2023Auxiliary Learning as an Asymmetric Bargaining Gamecited by 12
  3. 2023Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methodscited by 10
  4. 2021Linear Speedup in Personalized Collaborative Learningcited by 8
  5. 2022Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solutioncited by 6
  6. 2022Adaptive Mixing of Auxiliary Losses in Supervised Learningcited by 5
  7. 2022Image Classification with Small Datasets: Overview and Benchmarkcited by 4
  8. 2021Auxiliary Learning for Self-Supervised Video Representation via Similarity-based Knowledge Distillationcited by 3
  9. 2024Meta-Auxiliary Learning for Micro-Expression Recognitioncited by 3
  10. 2022Benchmarking Visual-Inertial Deep Multimodal Fusion for Relative Pose Regression and Odometry-aided Absolute Pose Regressioncited by 2
  11. 2025NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspectivecited by 2
  12. 2026FGTBT: Frequency-Guided Task-Balancing Transformer for Unified Facial Landmark Detectioncited by 2
  13. 2026Non-Stationary Functional Bilevel Optimizationcited by 2
  14. 2022Context Label Learning: Improving Background Class Representations in Semantic Segmentationcited by 1
  15. 2023No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasetscited by 1
  16. 2024Enhancing Molecular Property Prediction with Auxiliary Learning and Task-Specific Adaptationcited by 1
  17. 2024Robust Analysis of Multi-Task Learning Efficiency: New Benchmarks on Light-Weighed Backbones and Effective Measurement of Multi-Task Learning Challenges by Feature Disentanglementcited by 1
  18. 2024Data-Driven Preference Sampling for Pareto Front Learningcited by 1
  19. 2024Aux-NAS: Exploiting Auxiliary Labels with Negligibly Extra Inference Costcited by 1
  20. 2024Learning Representation for Multitask learning through Self Supervised Auxiliary learningcited by 1
  21. 2025Adapting Foundation Models for Few-Shot Medical Image Segmentation: Actively and Sequentiallycited by 1
  22. 2025Behavior Importance-Aware Graph Neural Architecture Search for Cross-Domain Recommendationcited by 1
  23. 2025Auxiliary Gene Learning: Spatial Gene Expression Estimation by Auxiliary Gene Selectioncited by 1
  24. 2023FedL2P: Federated Learning to Personalizecited by 0
  25. 2023Disentangled Latent Spaces Facilitate Data-Driven Auxiliary Learningcited by 0
  26. 2024Multi Task Inverse Reinforcement Learning for Common Sense Rewardcited by 0
  27. 2024Selecting the Best Sequential Transfer Path for Medical Image Segmentation with Limited Labeled Datacited by 0
  28. 2025Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learningcited by 0
  29. 2025Towards Fair and Robust Face Parsing for Generative AI: A Multi-Objective Approachcited by 0
  30. 2025Adversarial Attacks in Weight-Space Classifierscited by 0
  31. 2025Generating Auxiliary Tasks with Reinforcement Learningcited by 0
  32. 2025Learning Visual Affordance from Audiocited by 0
  33. 2026Feed m Birds with One Scone: Accelerating Multi-task Gradient Balancing via Bi-level Optimizationcited by 0