An Entropy Based Alternate Approach to Develop an Ensemble Classifier

R Aravind Sekhar; K G Sreeni

doi:10.17485/IJST/v16i33.85

Article

An Entropy Based Alternate Approach to Develop an Ensemble Classifier

VIEWS 373
PDF 88

Indian Journal of Science and Technology

DOI: 10.17485/IJST/v16i33.85

Year: 2023, Volume: 16, Issue: 33, Pages: 2601-2608

Original Article

An Entropy Based Alternate Approach to Develop an Ensemble Classifier

R Aravind Sekhar^1*, K G Sreeni¹

¹College of Engineering, Trivandrum, India

*Corresponding Author
Email: [email protected]
[email protected]

Received Date:12 January 2023, Accepted Date:08 July 2023, Published Date:01 September 2023

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

Objectives: To develop an alternative approach to improve the performance of ensemble classifiers. Methods: The Principle of Maximum Entropy and a precision based diversity measure are used to develop the model. Here joint entropy values are used which strongly reflect the combined uncertainties in the actual prediction and the prediction obtained during the training phase. Towards that purpose probabilistic confusion matrix has been used. The performance of the proposed method is tested against twenty different datasets. Each dataset is different from one another in terms of the number of classes and features. The results have been compared with several existing state of art ensemble methods uch as Bagging, AdaBoost, Naive Bayes (NB), Weighted Majority Vote (WMV) and TransEnsemble Classifier (TrEnL). Findings:The results show that the proposed method can select the best among the classifiers and perform better when compared to conventional classifiers. Novelty: The maximum entropy principle is applied at the final prediction probability matrix so that the method can be used as a generalized technique to improve the performance of ensemble classifiers.

Keywords: Ensemble; Maximum Entropy; Confusion Matrix; Weighted Entropy; Probabilistic Confusion Matrix

References

Mienye YID, Y S. A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects. 2022. Available from: https://ieeexplore.ieee.org/document/9893798
Yazdizadeh A, Patterson Z, Farooq B. Ensemble Convolutional Neural Networks for Mode Inference in Smartphone Travel Survey. IEEE Transactions on Intelligent Transportation Systems. 2020;21(6):2232–2239. Available from: https://doi.org/10.1109/TITS.2019.2918923
Jan MZ, Munoz JC, Ali MA. A novel method for creating an optimized ensemble classifier by introducing cluster size reduction and diversity. IEEE Transactions on Knowledge and Data Engineering. 2020;2022(7):1. Available from: https://doi.org/10.1109/TKDE.2020.3025173
Sridhar S, Sanagavarapu S. Detection and Prognosis Evaluation of Diabetic Retinopathy using Ensemble Deep Convolutional Neural Networks. 2020 International Electronics Symposium (IES). 2020. Available from: https://doi.org/10.1109/IES50839.2020.9231789
Sravanthi N, Venkat ML, Harshini S, Ashesh K. An Ensemble Approach to Predict Weather Forecast using Machine Learning. 2020 International Conference on Smart Electronics and Communication (ICOSEC). 2020. Available from: https://doi.org/10.1109/ICOSEC49089.2020.9215444
Tripathi MAP, Ansari TK, Gandhi. Ensemble Computational Intelligent for Insomnia Sleep Stage Detection via the Sleep ECG Signal. IEEE Access. 2022;10. Available from: https://doi.org/10.1109/ACCESS.2022.3212120
Wang K, Hu T, Zhao B. An Unsupervised Deep Neural Network Approach Based on Ensemble Learning to Suppress Seismic Surface-Related Multiples. IEEE Transactions on Geoscience and Remote Sensing. 2022;60:1–14. Available from: https://doi.org/10.1109/TGRS.2022.3225809
Tian Y, Zhang J, Chen Q, Liu Z. A Novel Selective Ensemble Learning Method for Smartphone Sensor-Based Human Activity Recognition Based on Hybrid Diversity Enhancement and Improved Binary Glowworm Swarm Optimization. IEEE Access. 2022;10:125027–125041. Available from: https://doi.org/10.1109/ACCESS.2022.3225652
Feng B, Gu Y, Yu H, Yang X, Gao S. DME: An Adaptive and Just-in-Time Weighted Ensemble Learning Method for Classifying Block-Based Concept Drift Steam. IEEE Access. 2022;10:120578–120591. Available from: https://doi.org/10.1109/ACCESS.2022.3222178
Zhai P, Hou T, Ji X, Dong Z, Zhang L. Robust Adaptive Ensemble Adversary Reinforcement Learning. IEEE Robotics and Automation Letters. 2022;7(4):12562–12568. Available from: https://doi.org/10.1109/LRA.2022.3220531
Xu H, Liu Q. An Ensemble Learning Algorithm Based on Density Peaks Clustering and Fitness for Imbalanced Data. IEEE Access. 2022;10:116120–116128. Available from: https://doi.org/10.1109/ACCESS.2022.3219582
Vinayagam A, Veerasamy V, Tariq M, Aziz A. Heterogeneous learning method of ensemble classifiers for identification and classification of power quality events and fault transients in wind power integrated microgrid. Sustainable Energy, Grids and Networks. 2022;31:100752. Available from: https://doi.org/10.1016/j.segan.2022.100752
Pamila JCMJ, Selvi RS, Santhi P, Nithya TM. Ensemble classifier based big data classification with hybrid optimal feature selection. Advances in Engineering Software. 2022;173:103183. Available from: https://doi.org/10.1016/j.advengsoft.2022.103183
Yang J, Wang F. Auto-Ensemble: An Adaptive Learning Rate Scheduling Based Deep Learning Model Ensembling. IEEE Access. 2020;8:217499–217509. Available from: https://arxiv.org/pdf/2003.11266.pdf#:~:text=Auto%2DEnsemble%20method%20refers%20to,generaliza%2D%20tion%20performance%20of%20classification.

Copyright

© 2023 Sekhar & Sreeni. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Published By Indian Society for Education and Environment (iSee)