# Application-Driven Innovation in Machine Learning

arXiv: https://arxiv.org/abs/2403.17381 · HTML: https://mlsanity.com/paper/2403.17381/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 59 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Builds on (most cited first, 30 of 31)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Multimodal datasets: misogyny, pornography, and malignant stereotypes](https://mlsanity.com/paper/2110.01963/) (2021)
- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2208.05419/) (2022)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery](https://mlsanity.com/paper/2311.17179/) (2023)
- [Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)](https://mlsanity.com/paper/2205.01397/) (2022)
- [Presence-Only Geographical Priors for Fine-Grained Image Classification](https://mlsanity.com/paper/1906.05272/) (2019)
- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [Lightweight, Pre-trained Transformers for Remote Sensing Timeseries](https://mlsanity.com/paper/2304.14065/) (2023)
- [FAENet: Frame Averaging Equivariant GNN for Materials Modeling](https://mlsanity.com/paper/2305.05577/) (2023)
- [A Review of Challenges and Opportunities in Machine Learning for Health](https://mlsanity.com/paper/1806.00388/) (2018)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [PASS: An ImageNet replacement for self-supervised pretraining without humans](https://mlsanity.com/paper/2109.13228/) (2021)
- [Field Study in Deploying Restless Multi-Armed Bandits: Assisting Non-Profits in Improving Maternal and Child Health](https://mlsanity.com/paper/2109.08075/) (2021)
- [Enforcing robust control guarantees within neural network policies](https://mlsanity.com/paper/2011.08105/) (2020)
- [Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization](https://mlsanity.com/paper/2105.08881/) (2021)
- [CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Management](https://mlsanity.com/paper/2012.10504/) (2020)
- [Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling](https://mlsanity.com/paper/2305.14452/) (2023)
- [Bugs in the Data: How ImageNet Misrepresents Biodiversity](https://mlsanity.com/paper/2208.11695/) (2022)
- [The Need for Ethical, Responsible, and Trustworthy Artificial Intelligence for Environmental Sciences](https://mlsanity.com/paper/2112.08453/) (2021)
- [Scalable Decision-Focused Learning in Restless Multi-Armed Bandits with Application to Maternal and Child Health](https://mlsanity.com/paper/2202.00916/) (2022)
- [TIML: Task-Informed Meta-Learning for Agriculture](https://mlsanity.com/paper/2202.02124/) (2022)

## Built on by (most cited first, 24 of 24)

- [The Landscape of Causal Discovery Data: Grounding Causal Discovery in Real-World Applications](https://mlsanity.com/paper/2412.01953/) (2024)
- [AstroPT: Scaling Large Observation Models for Astronomy](https://mlsanity.com/paper/2405.14930/) (2024)
- [DOLOMITES: Domain-Specific Long-Form Methodical Tasks](https://mlsanity.com/paper/2405.05938/) (2024)
- [IntersectionZoo: Eco-driving for Benchmarking Multi-Agent Contextual Reinforcement Learning](https://mlsanity.com/paper/2410.15221/) (2024)
- [SelvaBox: A high-resolution dataset for tropical tree crown detection](https://mlsanity.com/paper/2507.00170/) (2025)
- [SelvaMask: Segmenting Trees in Tropical Forests and Beyond](https://mlsanity.com/paper/2602.02426/) (2026)
- [Position: Biology is the Challenge Physics-Informed ML Needs to Evolve](https://mlsanity.com/paper/2510.25368/) (2025)
- [AI for Just Work: Constructing Diverse Imaginations of AI beyond "Replacing Humans"](https://mlsanity.com/paper/2503.08720/) (2025)
- [Position: We Need Responsible, Application-Driven (RAD) AI Research](https://mlsanity.com/paper/2505.04104/) (2025)
- [Fostering the Ecosystem of AI for Social Impact Requires Expanding and Strengthening Evaluation Standards](https://mlsanity.com/paper/2510.18238/) (2025)
- [MOMO: Mars Orbital Model Foundation Model for Mars Orbital Applications](https://mlsanity.com/paper/2604.02719/) (2026)
- [Improving satellite imagery segmentation using multiple Sentinel-2 revisits](https://mlsanity.com/paper/2409.17363/) (2024)
- [RL for Mitigating Cascading Failures: Targeted Exploration via Sensitivity Factors](https://mlsanity.com/paper/2411.18050/) (2024)
- [Towards Application-Specific Evaluation of Vision Models: Case Studies in Ecology and Biology](https://mlsanity.com/paper/2505.02825/) (2025)
- [Position: RL Researchers Need to Distinguish Between Solving Simulators and Using Simulators as a Proxy](https://mlsanity.com/paper/2606.28433/) (2026)
- [A Research Agenda for Usability and Generalisation in Reinforcement Learning](https://mlsanity.com/paper/2412.16970/) (2024)
- [Geospatial Artificial Intelligence for Satellite-Based Flood Extent Mapping: Concepts, Advances, and Future Perspectives](https://mlsanity.com/paper/2504.02214/) (2025)
- [kabr-tools: Automated Framework for Multi-Species Behavioral Monitoring](https://mlsanity.com/paper/2510.02030/) (2025)
- [Mass Conservation on Rails -- Rethinking Physics-Informed Learning of Ice Flow Vector Fields](https://mlsanity.com/paper/2510.06286/) (2025)
- [BioBench: A Blueprint to Move Beyond ImageNet for Scientific ML Benchmarks](https://mlsanity.com/paper/2511.16315/) (2025)
- [DynaSTy: A Framework for SpatioTemporal Node Attribute Prediction in Dynamic Graphs](https://mlsanity.com/paper/2601.05391/) (2026)
- [CHIRP dataset: towards long-term, individual-level, behavioral monitoring of bird populations in the wild](https://mlsanity.com/paper/2603.25524/) (2026)
- [AI Meets Mathematics Education: A Case Study on Supporting an Instructor in a Large Mathematics Class with Context-Aware AI](https://mlsanity.com/paper/2603.26679/) (2026)
- [Learning to Cover Locally: Graph Neural Combinatorial Optimization under a Hard Information Horizon](https://mlsanity.com/paper/2610.00422/) (2026)

## Abstract (arXiv)

In this position paper, we argue that application-driven research has been systemically under-valued in the machine learning community. As applications of machine learning proliferate, innovative algorithms inspired by specific real-world challenges have become increasingly important. Such work offers the potential for significant impact not merely in domains of application but also in machine learning itself. In this paper, we describe the paradigm of application-driven research in machine learning, contrasting it with the more standard paradigm of methods-driven research. We illustrate the benefits of application-driven machine learning and how this approach can productively synergize with methods-driven work. Despite these benefits, we find that reviewing, hiring, and teaching practices in machine learning often hold back application-driven innovation. We outline how these processes may be improved.