# Numerical differentiation of noisy data: A unifying multi-objective optimization framework

arXiv: https://arxiv.org/abs/2009.01911 · HTML: https://mlsanity.com/paper/2009.01911/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Sep 2020.

## Built on by (most cited first, 20 of 20)

- [Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization](https://mlsanity.com/paper/2206.00176/) (2022)
- [Benchmarking sparse system identification with low-dimensional chaos](https://mlsanity.com/paper/2302.10787/) (2023)
- [Discrepancy Modeling Framework: Learning missing physics, modeling systematic residuals, and disambiguating between deterministic and random effects](https://mlsanity.com/paper/2203.05164/) (2022)
- [Learning Anisotropic Interaction Rules from Individual Trajectories in a Heterogeneous Cellular Population](https://mlsanity.com/paper/2204.14141/) (2022)
- [Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art](https://mlsanity.com/paper/2412.01566/) (2024)
- [Sparse discovery of differential equations based on multi-fidelity Gaussian process](https://mlsanity.com/paper/2401.11825/) (2024)
- [Statistical Mechanics of Dynamical System Identification](https://mlsanity.com/paper/2403.01723/) (2024)
- [Uncovering Closed-form Governing Equations of Nonlinear Dynamics from Videos](https://mlsanity.com/paper/2106.04776/) (2021)
- [Baseline Drift Tolerant Signal Encoding for ECG Classification with Deep Learning](https://mlsanity.com/paper/2405.00724/) (2024)
- [Anomaly Detection in Complex Dynamical Systems: A Systematic Framework Using Embedding Theory and Physics-Inspired Consistency](https://mlsanity.com/paper/2502.19307/) (2025)
- [SODAs: Sparse Optimization for the Discovery of Differential and Algebraic Equations](https://mlsanity.com/paper/2503.05993/) (2025)
- [Hybrid Temporal Differential Consistency Autoencoder for Efficient and Sustainable Anomaly Detection in Cyber-Physical Systems](https://mlsanity.com/paper/2504.06320/) (2025)
- [Estimation Sample Complexity of a Class of Nonlinear Continuous-time Systems](https://mlsanity.com/paper/2312.05382/) (2023)
- [Automating the Discovery of Partial Differential Equations in Dynamical Systems](https://mlsanity.com/paper/2404.16444/) (2024)
- [Optimizing Hard Thresholding for Sparse Model Discovery](https://mlsanity.com/paper/2504.20256/) (2025)
- [A joint optimization approach to identifying sparse dynamics using least squares kernel collocation](https://mlsanity.com/paper/2511.18555/) (2025)
- [Learning piecewise-smooth dynamical systems](https://mlsanity.com/paper/2608.19785/) (2026)
- [Tensor-Train Weak SINDy: Identifying High-Dimensional Nonlinear Dynamics](https://mlsanity.com/paper/2609.09434/) (2026)
- [TinyUDE: Solver-Free Universal Differential Equations on Microcontrollers via Lie-Taylor Jet Matching](https://mlsanity.com/paper/2609.26972/) (2026)
- [AI-Assisted Tumor-Growth Law Discovery from Sparse Data: Refitting and LLM-Based Ranking](https://mlsanity.com/paper/2610.16891/) (2026)

## Abstract (arXiv)

Computing derivatives of noisy measurement data is ubiquitous in the physical, engineering, and biological sciences, and it is often a critical step in developing dynamic models or designing control. Unfortunately, the mathematical formulation of numerical differentiation is typically ill-posed, and researchers often resort to an textitad hoc process for choosing one of many computational methods and its parameters. In this work, we take a principled approach and propose a multi-objective optimization framework for choosing parameters that minimize a loss function to balance the faithfulness and smoothness of the derivative estimate. Our framework has three significant advantages. First, the task of selecting multiple parameters is reduced to choosing a single hyper-parameter. Second, where ground-truth data is unknown, we provide a heuristic for automatically selecting this hyper-parameter based on the power spectrum and temporal resolution of the data. Third, the optimal value of the hyper-parameter is consistent across different differentiation methods, thus our approach unifies vastly different numerical differentiation methods and facilitates unbiased comparison of their results. Finally, we provide an extensive open-source Python library textttpynumdiff to facilitate easy application to diverse datasets (https://github.com/florisvb/PyNumDiff).