Abstract In this paper, a new method for combining an ensemble of classifiers, called Consensus-based Combining Method (CCM) is proposed and evaluated. As in most other combination methods, the outputs of multiple classifiers are weighted and summed together into a single final classification decision. However, unlike the other methods, CCM adjusts the weights iteratively after comparing all of the classifiers’ outputs. Ultimately, all the weights converge to a final set of weights, and the combined output reaches a consensus. The effectiveness of CCM is evaluated by comparing it with popular linear combination methods (majority voting, product, and average method). Experiments are conducted on 14 public data sets, and on a blog spam data set created by the authors. Experimental results show that CCM provides a significant improvement in classification accuracy over the product and average methods. Moreover, results show that the CCM’s classification accuracy is better than or comparable to that of majority voting.

References

[1] Alzubi J., Diversity Based Improved Bagging Algorithm, in Proceedings of the International Conference on Engineering and MIS, Istanbul, 2015.

[2] Alzubi, J., Optimal Classifier Ensemble Design Based on Cooperative Game Theory, Research Journal of Applied Sciences, Engineering and Technology, vol. 11, no. 12, pp. 1336-1343, 2015.

[3] Alzubi O., An Empirical Study of Irregular AG Block Turbo Codes over Fading Channels, Research Journal of Applied Science Engineering and Technology, vol. 11, no. 12, pp. 1329-1335, 2015.

[4] Benediktsson J. and Swain P., Consensus Theoretic Classification Methods, IEEE Transactions on Systems, Man and Cybernetics, vol. 22, no. 4, pp. 688-704, 1992.

[5] Berenstein C., Kanal L., and Lavine D., Consensus Rules, in Proceedings of the Uncertainty in Artificial Intelligence Conference, Los Angeles, pp. 27-34, 1985.

[6] Berger R., A necessary and Sufficient Condition for Reaching a Consensus Using DeGroot's Method, Journal of the American Statistical Association, vol. 76, no. 374, pp. 415- 418,1981.

[7] DeGroot M., Reaching a Consensus, Journal of the American Statistical Association, vol. 69, no. 345, pp. 118-121, 1974.

[8] Fersini E., Messina E., and Pozzi F., Sentiment Analysis: Bayesian Ensemble Learning, Decision Support Systems, vol. 68, pp. 26-38, 2014.

[9] Hou S., Chen L., Tas E., Demihovskiy I., and Ye Y., Cluster-Oriented Ensemble Classifiers for Intelligent Malware Detection, in Proceeding of the International Conference on Semantic Computing, Anaheim, pp. 189-196, 2015.

[10] Kai L. and Ping Z., Using an Ensemble Classifier on Learning Evaluation for E-learning System, in Proceeding of in Computer Science and Service System, Nanjing, pp. 538-541, 2012.

[11] Kim Y. and Hong C., A Meta-Learning Approach for Combining Multiple Classifiers, Advanced Science and Technology Letters, vol. 29, pp. 50-54, 2013.

[12] Kuncheva L., Combining Pattern Classifiers: Methods and Algorithms, John Wiley and Sons, 2004.

[13] Leila C., Ma mar K., and Salim C., Combining Neural Networks for Arabic Handwriting Recognition, in Proceeding of International Symposium on Programming and Systems, Algiers, pp. 74-79, 2011.

[14] Li X., Liu X., and Yu L., Aggregative Model- Based Classifier Ensemble for Improving Land- Use/Cover Classification of Landsat TM Images, International Journal of Remote Sensing, vol. 35, no. 4, pp. 1481-1495, 2014.

[15] Liu M., Yuan B., Shu F., and Feng C., A new Classifier Combination Method Based on TSK- type Fuzzy System, in Proceding of International Conference on Wireless, Mobile and Multimedia Networks, Hangzhou, pp. 1-4, 2006. Consensus-Based Combining Method for Classifier Ensembles 85

[16] Newman J., Hettich S., Blake L., and Merz J., {UCI} Repository of Machine Learning Databases, Technical report 1998.

[17] Polikar R., Ensemble Based Systems in Decision Making, IEEE Circuits and Systems Magazine, vol. 6, no. 3, pp. 21-45, 2006.

[18] Quinlan J., Bagging, Boosting, and C4. 5, in Proceeding of the Thirteenth National Conference on Artificial Intelligence, Portland, pp. 725- 730,1996.

[19] Shaban K., Basir O., Kamel M., and Hassanein K., Intelligent Information Fusion Approach in Cooperative Multiagent Systems, in Proceeding of the 5th Biannual World Automation Congress, Orlando, pp. 429-434,2002.

[20] Tax D., Van Breukelen M., Duin R., and Kittler J., Combining Multiple Classifiers by Averaging or by Multiplying?, Pattern Recognition, vol. 33, no. 9, pp. 1475-1485, 2000.

[21] Toh K., Yau W., and Jiang X., A Reduced Multivariate Polynomial Model for Multimodal Biometrics and Classifiers Fusion, IEEE Transactions on Circuits and Systems for Video Technology, vol. 14, no. 2, pp. 224-233, 2004.

[22] Webb G., Multiboosting: A Technique for Combining Boosting and Wagging, Machine Learning, vol. 40, no. 2, pp. 159-196, 2000.

[23] Zhou Z., Ensemble Methods: Foundations and Algorithms, CRC Press, 2012.

[24] Zou Q., Guo J., Ju Y., Wu M., Zeng X., and HongZ., Improving TRNAscan SE Annotation Results Via Ensemble Classifiers, Molecular Informatics, vol. 34, no. 11 12, pp. 761-770, 2015. 86 The International Arab Journal of Information Technology, Vol. 15, No. 1, January 2018 Omar Alzubi was born in Allan, Jordan, in 1968. He received Master degree with distinction in Computer and Network Security from New York Institute of Technology (New York, USA) in 2006. He also holds Ph.D. degree in Computer and Network Security from Swansea University (Swansea, UK) in 2013.He joined Al-Balqa Applied University since 2013 as an assistant professor in computer and network security. Dr. Alzubi research interest includes network security, cloud security, application of Algebraic-Geometric theory in channel coding, machine learning, and Elliptic curve cryptosystems. He is also involved in UK-Turkey Higher Education Partnership Program 2011-2013 projects where he proposed a cryptosystem based on Elliptic curves. Jafar Alzubi received a B.Sc (Hons) in Electrical Engineering, majoring Electronics and Communications from the University of Engineering and Technology, Lahore, Pakistan in 2001. In 2005 received M.Sc. (Hons) in Electrical and Computer Engineering from New York Institute of Technology, New York, USA. Between 2005-2008, he became a full time lectures in the School of Engineering at Al-Balqa Applied University. In 2008, He joined the Wireless Communications Research Laboratory at Swansea University (Swansea, UK), completing his PhD in Advanced Telecommunications Engineering in June 2012. He is now an Assistant professor at Computer Engineering department, Al-Balqa Applied University; also he is deputy dean of Engineering Faculty. His research interests include Elliptic curves cryptography and cryptosystems, classifications, and coding. As part of his research, he designed the first regular and first irregular block turbo codes using Algebraic Geometry codes and investigated their performance across various computer and wireless networks. Sara Tedmori, in 2001, Dr. Tedmori received her BSc degree in Computer Science from the American University of Beirut, Lebanon. In 2003, she obtained her MSc degree in Multimedia and Internet Computing from Loughborough University. In 2008, she received her Engineering Doctorate in Computer Science from Loughborough University, UK. Currently she is an assistant professor in the Computer Science Department at Princess Sumaya University of Technology, Jordan. Her research interests include: sentiment analysis, image processing, knowledge extraction, classification, knowledge sharing, privacy, and software engineering. Hasan Rashaideh received his Bachelor and Master degrees in computer science and information technology from Yarmouk University in 1999 and 2002 respectively. In 2008 he obtained his PhD degree in computer science from Saint Petersburg Electrotechnical State University. Then after, he joined the department of computer science at Prince Abdullah Bin Ghazi Faculty of Information Technology / Al-Balqa Applied University- Jordan as an assistant professor, in 2015 he has been appointed as the Head of the department His research interests includes: machine learning, image processing and computer vision, information retrieval and optimization. Omar Almomani received his Bachelor and Master degree in Telecommunication Technology from institute of Information Technology, University of Sindh in 2002 and 2003 respectively. He received his PhD from University Utara Malaysia in computer network. Currently he is assistant professor and Vice Dean of Information Technology Faculty, the World Islamic Sciences & Education His research interests involves mobile ad hoc networks, Network Performance, Multimedia Networks, Network Quality of Service (QoS), IP Multicast, Network modelling and Simulation and Grid Computing.