mlsanityData to 7 Oct 2026

Compositional Risk Minimization

arXiv 2410.06303 ↗First posted 8 Oct 2024cs.LG cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Compositional generalization is a crucial step towards developing data-efficient intelligent machines that generalize in human-like ways. In this work, we tackle a challenging form of distribution shift, termed compositional shift, where some attribute combinations are completely absent at training but present in the test distribution. This shift tests the model's ability to generalize compositionally to novel attribute combinations in discriminative tasks. We model the data with flexible additive energy distributions, where each energy term represents an attribute, and derive a simple alternative to empirical risk minimization termed compositional risk minimization (CRM). We first train an additive energy classifier to predict the multiple attributes and then adjust this classifier to tackle compositional shifts. We provide an extensive theoretical analysis of CRM, where we show that our proposal extrapolates to special affine hulls of seen attribute combinations. Empirical evaluations on benchmark datasets confirms the improved robustness of CRM compared to other methods from the literature designed to tackle various forms of subpopulation shifts.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 41 of its 52 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

41 of 52 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2019Invariant Risk Minimizationcited by 1,832
  6. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  7. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  8. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 806
  9. 2021Domain Generalization: A Surveycited by 760
  10. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  11. 2020Long-tail learning via logit adjustmentcited by 605
  12. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 397
29 more
  1. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  2. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  3. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  4. 2020The Risks of Invariant Risk Minimizationcited by 262
  5. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 260
  6. 2021Simple data balancing achieves competitive worst-group-accuracycited by 177
  7. 2023Change is Hard: A Closer Look at Subpopulation Shiftcited by 119
  8. 2017Learning Independent Causal Mechanismscited by 104
  9. 2022NICO++: Towards Better Benchmarking for Domain Generalizationcited by 86
  10. 2022MetaShift: A Dataset of Datasets for Evaluating Contextual Distribution Shifts and Training Conflictscited by 81
  11. 2021Learning to Compose Visual Relationscited by 72
  12. 2021Unsupervised Learning of Compositional Energy Conceptscited by 71
  13. 2021Controllable and Compositional Generation with Latent-Space Energy-Based Modelscited by 64
  14. 2020Empirical or Invariant Risk Minimization? A Sample Complexity Perspectivecited by 62
  15. 2023Compositional Generalization from First Principlescited by 62
  16. 2023Concept Algebra for (Score-Based) Text-Controlled Generative Modelscited by 59
  17. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  18. 2023Provably Learning Object-Centric Representationscited by 45
  19. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  20. 2022Avoiding spurious correlations via logit correctioncited by 32
  21. 2023Discovering environments with XRMcited by 26
  22. 2023Group Robust Classification Without Any Group Informationcited by 25
  23. 2019Compositional Generalization in Image Captioningcited by 23
  24. 2023Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learningcited by 22
  25. 2023Discovering modular solutions that generalize compositionallycited by 22
  26. 2022First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domainscited by 20
  27. 2024A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practicescited by 15
  28. 2023A Survey on Compositional Generalization in Applicationscited by 12
  29. 2024Compositional Image Decomposition with Diffusion Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 5.

13 papers.

Sort:

By area:

  1. 2025Generative Trajectory Stitching through Diffusion Compositioncited by 38
  2. 2025Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Researchcited by 9
  3. 2025Product of Experts for Visual Generationcited by 6
  4. 2026Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisationcited by 5
  5. 2025Provable Benefits of In-Tool Learning for Large Language Modelscited by 4
  6. 2025Compositional Scene Understanding through Inverse Generative Modelingcited by 3
  7. 2026Compositional Visual Planning via Inference-Time Diffusion Scalingcited by 2
  8. 2025Ctrl-VI: Controllable Video Synthesis via Variational Inferencecited by 1
  9. 2026Long-Text-to-Image Generation via Compositional Prompt Decompositioncited by 1
  10. 2026Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglementcited by 1
  11. 2025Compositional World Knowledge leads to High Utility Synthetic datacited by 0
  12. 2025A Theoretical Analysis of Provable Compositional Generalization in Neural Networks: A Necessary and Sufficient Conditioncited by 0
1 more
  1. 2026Who Needs Labels? Adapting Vision Foundation Models With the Metadata You Already Havecited by 0