# Supervised feature selection with orthogonal regression and feature weighting

arXiv: https://arxiv.org/abs/1910.03787 · HTML: https://mlsanity.com/paper/1910.03787/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2019.

## Built on by (most cited first, 6 of 6)

- [Automatic feature selection and weighting in molecular systems using Differentiable Information Imbalance](https://mlsanity.com/paper/2411.00851/) (2024)
- [Embedded Multi-label Feature Selection via Orthogonal Regression](https://mlsanity.com/paper/2403.00307/) (2024)
- [Low-Dimensional Representation-Driven TSK Fuzzy System for Feature Selection](https://mlsanity.com/paper/2501.12607/) (2025)
- [ADSEL: Adaptive Dual Self-Expression Learning for EEG Feature Selection via Incomplete Multi-Dimensional Emotion Labels](https://mlsanity.com/paper/2508.05229/) (2025)
- [REFS: Robust EEG feature selection with missing multi-dimensional annotation for emotion recognition](https://mlsanity.com/paper/2508.05933/) (2025)
- [Incomplete Depression Feature Selection with Missing EEG Channels](https://mlsanity.com/paper/2511.11651/) (2025)

## Abstract (arXiv)

Effective features can improve the performance of a model, which can thus help us understand the characteristics and underlying structure of complex data. Previous feature selection methods usually cannot keep more local structure information. To address the defects previously mentioned, we propose a novel supervised orthogonal least square regression model with feature weighting for feature selection. The optimization problem of the objection function can be solved by employing generalized power iteration (GPI) and augmented Lagrangian multiplier (ALM) methods. Experimental results show that the proposed method can more effectively reduce the feature dimensionality and obtain better classification results than traditional feature selection methods. The convergence of our iterative method is proved as well. Consequently, the effectiveness and superiority of the proposed method are verified both theoretically and experimentally.