Analysis of an Educational dataset using Classification Algorithm in Conjunction with Wrapper Feature Selection Methods analysis-of-an-educational-dataset-using-classification-algorithm-in-conjunction-with-wrapper-feature-selection-methods Volume 5, Issue 2 ARTICLES 2025 5 Feature selection plays a critical role in improving the efficiency, accuracy, and interpretability of machine learning models, particularly when dealing with high-dimensional datasets. Among various approaches, wrapper-based feature selection methods are known for their ability to capture feature interactions by directly optimizing model performance. This study presents a comprehensive comparative analysis of six wrapper feature selection techniques—Recursive Feature Elimination (RFE), Sequential Forward Selection (SFS), Sequential Backward Selection (SBS), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Differential Evolution (DE)—in conjunction with five widely used classification algorithms: Decision Tree, K-Nearest Neighbour, Random Forest, Logistic Regression, and Support Vector Machine. Experiments are conducted on an educational dataset comprising 395 student records with 30 attributes obtained from the UCI repository, using different feature subset sizes (all features, top 20, top 15, and top 10). Model performance is evaluated using accuracy, precision, recall, F1-score, and AUC. The results demonstrate that wrapper methods significantly enhance classification performance while reducing dimensionality, with GA and RFE consistently emerging as the most effective techniques across multiple classifiers. DE also shows strong performance, particularly with Logistic Regression and Random Forest, whereas PSO generally underperforms in terms of AUC. Furthermore, reducing the feature set does not adversely affect predictive accuracy and, in several cases, leads to improved generalization. The findings confirm the effectiveness of wrapper methods for educational data mining and provide practical insights for selecting optimal feature–classifier combinations.

]]> Asif, R., Merceron, A., & Pathan, M. K. (2017). Predicting student academic performance at degree level: A case study. International Journal of Intelligent Systems and Applications, 9(12), 49–61. https://doi.org/10.5815/ijisa.2017.12.06

 

Bin Mat, U., Buniyamin, N., Arsad, P. M., & Kassim, R. A. (2014). An overview of using academic analytics to predict and improve students’ achievement: A proposed proactive intelligent intervention. In Proceedings of the 2013 IEEE 5th International Conference on Engineering Education (ICEED) (pp. 126–130). IEEE.

 

Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16–28.

 

Das, S., & Suganthan, P. N. (2011). Differential evolution: A survey of the state-of-the-art. IEEE Transactions on Evolutionary Computation, 15(1), 4–31. https://doi.org/10.1109/ TEVC.2010.2059031

 

Dash, M., & Liu, H. (1997). Feature selection for classification. Intelligent Data Analysis, 1(3), 131–156.

 

Dubois, P. F. (2007). Python: Batteries included. Computing in Science & Engineering, 9(3), 7–9. https://doi.org/10.1109/MCSE.2007.58

 

Feng, C., & Xu, X. (2019). A hybrid feature selection method based on RFE and Relief for educational data mining. Computers in Human Behavior, 101, 228–237. https://doi.org/10.1016/j. chb.2019.07.002

 

Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Addison-Wesley.

 

Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157–1182.

 

 

 

Mamta Saxena, Netra Pal Singh

 

Analysis of an Educational dataset using Classification Algorithm in Conjunction with Wrapper Feature Selection Methods

 

Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002). Gene selection for cancer classification using support vector machines. Machine Learning, 46(1–3), 389–422. https://doi.org/10.1023/A:1012487302797

 

Holland, J. H. (1992). Adaptation in natural and artificial systems: An introductory analysis with applications to biology, control, and artificial intelligence. MIT Press.

 

Islam, M. M., & Murase, K. (2018). A hybrid wrapper–filter approach for feature selection using mutual information and differential evolution. Knowledge-Based Systems, 147, 125–134. https://doi.org/10.1016/j.knosys.2018.01.002

 

Jain, A. K., & Zongker, D. (1997). Feature selection: Evaluation, application, and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(2), 153–158. https://doi.org/10.1109/34.574797

 

Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of the IEEE International Conference on Neural Networks (pp. 1942–1948). IEEE.

 

Khan, M. J., & Jawaid, A. (2020). Prediction of at-risk students using genetic algorithm and random forest. Education and Information Technologies, 25(4), 2979–2997.

 

Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97(1–2), 273–324.

 

Li, J., Cheng, K., Wang, S., Morstatter, F., Trevino, R. P., Tang, J., & Liu, H. (2017). Feature selection: A data perspective. ACM Computing Surveys, 50(6), 1–45.

 

Liu, H., & Yu, L. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4), 491–502. https:// doi.org/10.1109/TKDE.2005.66

 

Maldonado, S., & Weber, R. (2009). A wrapper method for feature selection using support vector machines. Information Sciences, 179(13), 2208–2217.

 

Millman, K. J., & Aivazis, M. A. (2011). Python for scientists and engineers. Computing in Science & Engineering, 13(2), 9–12. https://doi.org/10.1109/MCSE.2011.36

 

Moslehi, F., & Haeri, A. (2020). A novel hybrid wrapper–filter approach based on genetic algorithm and particle swarm optimization for feature subset selection. Journal of Ambient Intelligence and Humanized Computing, 11, 1105–1127.

 

Patel, D., Saxena, A., & Wang, J. (2024). A machine-learning-based wrapper method for feature selection. International Journal of Data Warehousing and Mining. https://doi.org/10.4018/ IJDWM.352041

 

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., … Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.

 

Pudil, P., Novovicová, J., & Kittler, J. (1994). Floating search methods in feature selection. Pattern Recognition Letters, 15(11), 1119–1125. https://doi.org/10.1016/0167-8655(94)90127-9

 

Saeys, Y., Inza, I., & Larrañaga, P. (2007). A review of feature selection techniques in bioinformatics. Bioinformatics, 23(19), 2507–2517.

 

 

Saxena, M., & Singh, N. P. (2025). Role of Hybrid Feature Selection Algorithms in Foreseeing Stu dent Performance. International Journal of Natural and Technical Sciences, 5(1), 43-75. https://doi.org/10.69648/NVAX8158

 

Sharma, M., & Kaur, P. (2020). An efficient differential evolution approach for feature selection in educational data mining. International Journal of Cognitive Computing in Engineering, 1, 33–43.

 

Siedlecki, W., & Sklansky, J. (1989). A note on genetic algorithms for large-scale feature selection. Pattern Recognition Letters, 10(5), 335–347.

 

Singh, A. K., & Karthikeyan, S. (2024). A wrapper feature selection method for predicting student dropout in higher education. SSRN Electronic Journal. https://doi.org/10.2139/ ssrn.5002077

 

Sivanandam, S. N., & Deepa, S. N. (2007). Introduction to genetic algorithms. Springer. https://doi. org/10.1007/978-3-540-73190-0

 

Storn, R., & Price, K. (1997). Differential evolution – A simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.

 

Tang, J., Alelyani, S., & Liu, H. (2014). Feature selection for classification: A review. In H. Liu & L. Yu (Eds.), Data classification: Algorithms and applications (pp. 37–64). CRC Press.

 

Tomasevic, N., Gvozdenovic, N., & Vranes, S. (2019). An overview and comparison of supervised data mining techniques for student exam performance prediction. Computers & Education, 143, 103676. https://doi.org/10.1016/j.compedu.2019.103676

 

Waheed, H., Goyal, M., & Khanum, A. (2020). Machine learning-based early prediction of students’ performance using PSO. Education and Information Technologies, 25(6), 4693–4711.

 

Xue, B., Zhang, M., & Browne, W. N. (2015). A comprehensive comparison on evolutionary feature selection approaches to classification. International Journal of Computational Intelligence and Applications, 14(2), 1550008.

 

Xue, B., Zhang, M., & Browne, W. N. (2016). A survey on evolutionary computation appr

]]> Classification Algorithms, Feature Selection, Performance Metrics, Wrapper Methods https://ijtns.ibupress.com/articles/analysis-of-an-educational-dataset-using-classification-algorithm-in-conjunction-with-wrapper-feature-selection-methods