A Hybrid Intersection Filtering and Recursive Feature Elimination Technique for Efficient Feature Reduction in High Dimensional Datasets

Authors

  • Akhmad Dahlan Universitas Amikom Yogyakarta
  • Yoga Pristyanto Universitas Amikom Yogyakarta
  • Anggit Ferdita Nugraha Universitas Amikom Yogyakarta
  • Rifda Faticha Alfa Aziza Universitas Amikom Yogyakarta
  • Ibnu Hadi Purwanto Universitas Amikom Yogyakarta

DOI:

https://doi.org/10.29207/resti.v10i2.7396

Keywords:

classification, feature selection, intersection filter, machine learning, RFECV

Abstract

High-dimensional datasets are commonly encountered in real-world machine learning applications and often degrade classification performance due to redundant and irrelevant features. In addition, the presence of excessive features increases computational complexity and processing time. Feature selection is therefore a crucial preprocessing step to improve model accuracy and efficiency. This study proposes a hybrid feature selection approach called Intersection Filtering based on Recursive Feature Elimination with Cross-Validation (IF-RFECV), which integrates wrapper-based and filter-based strategies to obtain a stable and optimal subset of features. The proposed method first applies Recursive Feature Elimination with Cross-Validation (RFECV) using multiple classification models to rank and select relevant features. Subsequently, an intersection filtering mechanism is employed to identify features that are consistently selected across different RFECV-based models, thereby reducing model-dependent bias and improving feature robustness. The effectiveness of IF-RFECV is evaluated using four benchmark datasets with varying dimensionality obtained from the KEEL and UCI repositories. Several classification algorithms, including Gradient Boosting, K-Nearest Neighbor, Naïve Bayes, Decision Tree, Random Forest, and Support Vector Machine, are used to assess model performance. Experimental results demonstrate that IF-RFECV produces a more compact feature subset compared to conventional RFECV while achieving superior performance in terms of accuracy, precision, recall, and F1-score on most datasets, particularly those with higher dimensionality. Although IF-RFECV requires slightly higher computational time due to its two-stage process, the performance gains and improved generalization justify this trade-off. These findings indicate that IF-RFECV is an effective and robust feature selection technique for high-dimensional classification problems.

Downloads

Download data is not yet available.

References

N. Almusallam, Z. Tari, J. Chan, A. Fahad, A. Alabdulatif, and M. Al-Naeem, “Towards an Unsupervised Feature Selection Method for Effective Dynamic Features,” IEEE Access, vol. 9, pp. 77149–77163, 2021, doi: 10.1109/ACCESS.2021.3082755.

R. C. Chen, C. Dewi, S. W. Huang, and R. E. Caraka, “Selecting critical features for data classification based on machine learning methods,” J Big Data, vol. 7, no. 1, 2020, doi: 10.1186/s40537-020-00327-4.

R. Islam, M. I. Sayed, S. Saha, M. J. Hossain, and M. A. Masud, “Android malware classification using optimum feature selection and ensemble machine learning,” Internet of Things and Cyber-Physical Systems, vol. 3, no. January, pp. 100–111, 2023, doi: 10.1016/j.iotcps.2023.03.001.

P. Morán, A. Robles-Gómez, A. Duque, L. Tobarra, and R. Pastor-Vargas, “Machine learning models and dimensionality reduction for improving the Android malware detection,” PeerJ Comput Sci, vol. 10, pp. 1–31, 2024, doi: 10.7717/PEERJ-CS.2616.

R. Islam, M. I. Sayed, S. Saha, M. J. Hossain, and M. A. Masud, “Android malware classification using optimum feature selection and ensemble machine learning,” Internet of Things and Cyber-Physical Systems, vol. 3, no. October 2022, pp. 100–111, 2023, doi: 10.1016/j.iotcps.2023.03.001.

A. Pathak, U. Barman, and T. S. Kumar, “Machine learning approach to detect android malware using feature-selection based on feature importance score,” Journal of Engineering Research (Kuwait), no. March, 2024, doi: 10.1016/j.jer.2024.04.008.

A. Newaz and S. Muhtadi, “Performance Improvement of Heart Disease Prediction by Identifying Optimal Feature Sets Using Feature Selection Technique,” in 2021 International Conference on Information Technology (ICIT), 2021, pp. 446–450. doi: 10.1109/ICIT52682.2021.9491739.

R. Islam, M. I. Sayed, S. Saha, M. J. Hossain, and M. A. Masud, “Android malware classification using optimum feature selection and ensemble machine learning,” Internet of Things and Cyber-Physical Systems, vol. 3, no. January, pp. 100–111, 2023, doi: 10.1016/j.iotcps.2023.03.001.

N. Almusallam, Z. Tari, J. Chan, A. Fahad, A. Alabdulatif, and M. Al-Naeem, “Towards an Unsupervised Feature Selection Method for Effective Dynamic Features,” IEEE Access, vol. 9, pp. 77149–77163, 2021, doi: 10.1109/ACCESS.2021.3082755.

E. Strelcenia and S. Prakoonwit, “Effective Feature Engineering and Classification of Breast Cancer Diagnosis: A Comparative Study,” BioMedInformatics, vol. 3, no. 3, pp. 616–631, 2023, doi: 10.3390/biomedinformatics3030042.

D. Ö. Şahin, O. E. Kural, S. Akleylek, and E. Kılıç, “Permission-based Android malware analysis by using dimension reduction with PCA and LDA,” Journal of Information Security and Applications, vol. 63, no. October, p. 102995, 2021, doi: 10.1016/j.jisa.2021.102995.

C. Marquez-Vera, A. Cano, C. Romero, and S. Ventura, “Predicting student failure at school using genetic programming and different data mining approaches with high dimensional and imbalanced data,” Applied Intelligence, vol. 38, no. 3, pp. 315–330, 2013, doi: 10.1007/s10489-012-0374-8.

Khadijah, A. Adorada, P. W. Wirawan, and K. Kurniawan, “The Comparison of Feature Selection Methods in Software Defect Prediction,” ICICoS 2020 - Proceeding: 4th International Conference on Informatics and Computational Sciences, pp. 0–5, 2020, doi: 10.1109/ICICoS51170.2020.9299022.

S. A. Roseline and S. Geetha, “Android Malware Detection and Classification using LOFO Feature Selection and Tree-based Models,” J Phys Conf Ser, vol. 1911, no. 1, 2021, doi: 10.1088/1742-6596/1911/1/012031.

M. Riajuliislam, K. Z. Rahim, and A. Mahmud, “Prediction of Thyroid Disease(Hypothyroid) in Early Stage Using Feature Selection and Classification Techniques,” 2021 International Conference on Information and Communication Technology for Sustainable Development, ICICT4SD 2021 - Proceedings, pp. 60–64, 2021, doi: 10.1109/ICICT4SD50815.2021.9397052.

S. Murugesan, R. S. Bhuvaneswaran, H. Khanna Nehemiah, S. Keerthana Sankari, and Y. Nancy Jane, “Feature Selection and Classification of Clinical Datasets Using Bioinspired Algorithms and Super Learner,” Comput Math Methods Med, vol. 2021, 2021, doi: 10.1155/2021/6662420.

Y. Yin et al., “IGRF-RFE: a hybrid feature selection method for MLP-based network intrusion detection on UNSW-NB15 dataset,” J Big Data, vol. 10, no. 1, 2023, doi: 10.1186/s40537-023-00694-8.

A. Gupta, A. Bhatt, V. P. Singh, A. Mathur, A. V Waghmare, and S. Joglekar, “Disturbance Detection in Power Quality Data using Principle Component Analysis, Recursive Feature Elimination and Logistic Regression,” in 2025 IEEE 5th International Conference on Sustainable Energy and Future Electric Transportation (SEFET), Jul. 2025, pp. 1–4. doi: 10.1109/SEFET65155.2025.11255423.

M. Riajuliislam, K. Z. Rahim, and A. Mahmud, “Prediction of Thyroid Disease(Hypothyroid) in Early Stage Using Feature Selection and Classification Techniques,” 2021 International Conference on Information and Communication Technology for Sustainable Development, ICICT4SD 2021 - Proceedings, pp. 60–64, 2021, doi: 10.1109/ICICT4SD50815.2021.9397052.

Y. Deng, Y. Wei, and Y. Li, “Credit Risk Evaluation Based on Data Mining and Integrated Feature Selection,” ICSPCC 2020 - IEEE International Conference on Signal Processing, Communications and Computing, Proceedings, pp. 8–11, 2020, doi: 10.1109/ICSPCC50002.2020.9259483.

Y. Deng, Y. Wei, and Y. Li, “Credit Risk Evaluation Based on Data Mining and Integrated Feature Selection,” ICSPCC 2020 - IEEE International Conference on Signal Processing, Communications and Computing, Proceedings, pp. 8–11, 2020, doi: 10.1109/ICSPCC50002.2020.9259483.

M. Chemmakha, O. Habibi, and M. Lazaar, “Improving Machine Learning Models for Malware Detection Using Embedded Feature Selection Method,” IFAC-PapersOnLine, vol. 55, no. 12, pp. 771–776, 2022, doi: 10.1016/j.ifacol.2022.07.406.

K. M. Sujon, I. U. Ahmed, M. M. I. Molla, N. Jahan, and S. K. Utsho, “The Impact of Recursive Feature Elimination and Information Gain on Machine Learning Models for Student Performance Prediction,” in 2025 International Conference on Quantum Photonics, Artificial Intelligence, and Networking (QPAIN), Jul. 2025, pp. 1–6. doi: 10.1109/QPAIN66474.2025.11172062.

S. K and J. A. Arul Jothi, “Disease Prediction and Biomarker Identification using Recursive Feature Elimination and Deep Forest classifier from Human Gut Metagenomic Data,” in 2025 IEEE International Symposium on Technology and Society (ISTAS), 2025, pp. 1–7. doi: 10.1109/ISTAS65609.2025.11269618.

C. Yi, M. Guo, Y. Lei, and T. Qiu, “Research on the Application of Ensemble Learning Model Based on Recursive Feature Elimination in Engine Efficiency Prediction,” in 2025 11th International Conference on Energy Materials and Electrical Engineering (ICEMEE), Aug. 2025, pp. 425–429. doi: 10.1109/ICEMEE66610.2025.11276539.

Arini, Hidayat, and A. Fiade, “Performance Evaluation of RFE-CV, Class Weights, and ADASYN for Network Attack Classification on UNSW-NB15 Using Ensemble Learning,” in 2025 13th International Conference on Cyber and IT Service Management (CITSM), 2025, pp. 1–7. doi: 10.1109/CITSM67730.2025.11291428.

Downloads

Published

2026-04-26

Issue

Section

Artificial Intelligence

How to Cite

Dahlan, A. ., Pristyanto, Y. ., Nugraha, A. F. ., Aziza, R. F. A. ., & Purwanto, I. H. . (2026). A Hybrid Intersection Filtering and Recursive Feature Elimination Technique for Efficient Feature Reduction in High Dimensional Datasets. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 10(2), 465-475. https://doi.org/10.29207/resti.v10i2.7396