In this paper, we propose an efficient framework to improve accuracy and computational cost of a Facial Expression Recognition (FER) system. This framework is carried out in three stages. In the initial one, corresponding to feature extraction, three descriptors, derived from Discrete Wavelet Transform (DWT), are introduced to extract distinct feature types. In the second stage, focused on feature selection, a Wrapper approach is adopted to carefully select the most relevant features from the previously extracted pool. Following feature selection, the Support Vector Machine (SVM) classifier is employed, in the final stage, to determine an individual's affective state. The experiments were conducted in person-independent mode using both the Japanese Female Facial Expression (JAFFE) and extended Cohn-Kanade (CK+) databases which included the following emotions: anger, disgust, contempt, fear, happy, sad, surprise, and neutral. The obtained results demonstrated the effectiveness of the proposed framework in increasing recognition rate and decreasing response time compared to other state-of-the-art methods. A comparative study between our proposed framework and that based on the Local Binary Patterns (LBP) method demonstrated that our framework outperforms the latter for most emotions. In fact, our proposed framework converges rapidly and achieves good performance, thus allowing us to develop a real-time Facial Expression Recognition (FER) system in person- independent mode. Average recognition rates of 89.66% and 87.76% were obtained using our method with the JAFFE database and the CK+ database, respectively.

[1]    Abdulrahman M., Gwadabe T., Abdu F., and Eleyan A., “Gabor Wavelet Transform Based Facial Expression Recognition Using PCA and LBP,” in Proceedings of the 22nd Signal Processing and Communications Applications Conference, Trabzon, pp. 2265-2268, 2014. doi:10.1109/SIU.2014.6830717
[2]    Abdulrazaq M., Mahmood M., Zeebaree S., Abdulwahab M., and Zebari, R., “An Analytical Appraisal for Supervised Classifiers’ Performance on Facial Expression Recognition Based on Relief-F Feature Selection,” in Proceedings of the International Conference of Modern Applications on Information and Communication Technology, Babylon-Hilla, pp. 012055, 2021. DOI:10.1088/1742-6596/1804/1/012055
[3]    Ahmed F., Bari H., and Hossain E., “Person-Independent Facial Expression Recognition Based on Compound Local Binary Pattern (CLBP),” The International Arab Journal of Information Technology, vol. 11, no. 2, pp. 195-203, 2014.
[4]    Ahmed N., Natarajan T., and Rao K. “Discrete Cosine Transform,” IEEE Transactions on Computers, vol. 100, no. 1, pp. 90-93, 1974, doi:10.1109/T-C.1974.223784
[5]    Al-Qablan T., Noor M., Al-Betar M., and Khader A., “Improved Binary Gray Wolf Optimizer Based on Adaptive Β-Hill Climbing for Feature Selection,” IEEE Access, vol. 11, pp. 59866-59881, 2023. doi:10.1109/ACCESS.2023.3285815
[6]    Bengacemi H., Hacine-Gharbi A., Ravier P., Abed-Meraim K., and Buttelli O., “Surface EMG Signal Classification for Parkinson’s Disease Using WCC Descriptor and ANN Classifier,” in Proceedings of the 10th International Conference on Pattern Recognition Applications and Methods, Porto, pp. 287-294, 2021. DOI:10.5220/0010254402870294
[7]    Clavel C. and Ehrette T., “Fear-Type Emotion Recognition and Abnormal Events Detection for an Audio-Based Surveillance System,” WIT Transactions on Information and Communication Technologies, vol. 39, pp. 471-479, 2008. DOI:10.2495/RISK080461
[8]    Dalal N. and Triggs B., “Histograms of Oriented Gradients for Human Detection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, pp. 886-893, 2005. doi:10.1109/CVPR.2005.177
[9]    Daren H., Jiufen L., Jiwu H., and Hongmei L., “A DWT-Based Image Watermarking Algorithm,” in Proceedings of the IEEE International Conference on Multimedia and Expo, Tokyo, pp. 313-316, 2001. doi:10.1109/ICME.2001.1237719
[10]    Didiot E., Illina I., Fohr D., and Mella O., “A Wavelet-based Parameterization for Speech/Music Discrimination,” Computer Speech and Language, vol. 24, no. 2, pp. 341-357, 2010. https://doi.org/10.1016/j.csl.2009.05.003
[11]    Dino H., Abdulrazzaq M., Zeebaree S., Sallow A., and Zebari R., “Facial Expression Recognition based on Hybrid Feature Extraction Techniques with Different Classifiers,” TEST Engineering and Management, vol. 83, pp. 22319-22329, 2020.
[12]    Dosodia P., Poonia A., Gupta S., and Agrwal S., “New Gabor-DCT Feature Extraction Technique for Facial Expression Recognition,” in Proceedings of the 5th International Conference on Communication Systems and Network Technologies, Gwalior, pp. 546-549, 2015. doi:10.1109/CSNT.2015.162
[13]    Gabor D., “Theory of Communication. Part 1: The Analysis of Information,” Journal of the Institution of Electrical Engineers-part III: Radio and Communication Engineering, vol. 93, no. 26, pp. 429-441, 1946. DOI:10.1049/ji-3-2.1946.0074
[14]    Ghosh M., Kundu T., Ghosh D., and Sarkar R., “Feature Selection for Facial Emotion Recognition Using Late Hill-Climbing Based Memetic Algorithm,” Multimedia Tools and Applications, vol. 78, pp. 25753-25779, 2019. https://doi.org/10.1007/s11042-019-07811-x
[15]    Hacine-Gharbi A., Ravier P., “Wavelet Cepstral Coefficients for Electrical Appliances Identification using Hidden Markov Models,” in Proceedings of the 7th International Conference on Pattern Recognition Applications and Methods, Madeira, pp. 541-549, 2018. DOI:10.5220/0006662305410549
[16]    Hacine-Gharbi A., Petit M., Ravier P., and Némo F., “Prosody based Automatic Classification of the Uses of French ‘Oui’ as Convinced or Unconvinced Uses,” in Proceedings of the International Conference on Pattern Recognition Applications and Methods, Lisbon, pp.349-354, 2015. DOI:10.5220/0005293103490354
[17]    Huang Y., Yan Y., Chen S., and Wang H., “Expression-Targeted Feature Learning for Effective Facial Expression Recognition,” Journal of Visual Communication and Image Representation, vol. 55, pp. 677-687, 2018. https://doi.org/10.1016/j.jvcir.2018.08.002
[18]    Kar N., Babu K., and Jena S., “Face Expression Recognition Using Histograms of Oriented Gradients with Reduced Features,” in Proceedings of the International Conference on Computer Vision and Image Processing, Uttarakhand, pp. 209-2019, 2016.
[19]    Karataş F., Koyuncu I., Tuna M., Alçın M., and Avcioglu E., “Design and Implementation of Arrhythmic ECG Signals for Biomedical Engineering Applications on FPGA,” European Physical Journal Special Topics, vol. 231, no. 5, pp. 869-884, 2022. https://doi.org/10.1140/epjs/s11734-021-00334-3

[20]    Kohavi R., and John G., “Wrappers for Feature Subset Selection,” Artificial Intelligence, vol. 97, no. 1-2, pp. 273-324, 1997. https://doi.org/10.1016/S0004-3702(97)00043-X
[21]    Kołakowska A., Landowska A., Szwoch M., Szwoch W., and Wrobel M., Human-Computer Systems Interaction: Backgrounds and Applications, Springer, 2014. https://doi.org/10.1007/978-3-319-08491-6_5
[22]    Lajevardi S. and Hussain Z., “Facial Expression Recognition Using Log-Gabor Filters and Local Binary Pattern Operators,” in Proceedings of the International Conference on Communication, Computer and Power, Muscat, pp. 349-353, 2009.
[23]    Lajevardi S. and Hussain Z., “Feature Selection for Facial Expression Recognition Based on Optimization Algorithm,” in Proceedings of the 2nd International Workshop on Nonlinear Dynamics and Synchronization, Klagenfurt, pp. 182-185, 2009. doi:10.1109/INDS.2009.5228001
[24]    Lajevardi S. and Hussain Z., “Feature Extraction for Facial Expression Recognition Based on Hybrid Face Regions,” Advances in Electrical and Computer Engineering, vol. 9, no. 3, pp. 63-67, 2009. DOI:10.4316/aece.2009.03012
[25]    Lei L. and Kun S., “Speaker Recognition Using Wavelet Cepstral Coefficient, I-Vector, and Cosine Distance Scoring and its Application for Forensics,” Journal of Electrical and Computer Engineering, vol. 2016, 2016. https://doi.org/10.1155/2016/4908412
[26]    Liu Q., Chen C., Zhang Y., and Hu Z., “Feature Selection for Support Vector Machines with RBF Kernel,” Artificial Intelligence Review, vol. 36, no. 2, pp. 99-115, 2011. https://doi.org/10.1007/s10462-011-9205-2
[27]    Lowe D., “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, vol. 60, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
[28]    Lucey P., Cohn J., Kanade T., Saragih J., and Ambadar Z., “The Extended Cohn-Kanade Dataset (CK+): A Complete Dataset for Action Unit and Emotion-Specified Expression,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, pp. 94-101, 2010. doi: 10.1109/CVPRW.2010.5543262
[29]    Lyons M., Akamatsu S., Kamachi M., and Gyoba J., “Coding Facial Expressions with Gabor Wavelets,” in Proceedings of the 3rd IEEE International Conference on Automatic Face and Gesture Recognition, Nara, pp. 200-205, 1998. doi: 10.1109/AFGR.1998.670949
[30]    Mallat S., “A Theory for Multiresolution Signal Decomposition: The Wavelet Representation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 11, no. 7, pp. 674-693, 1989. doi: 10.1109/34.192463
[31]    Mallat S., A Wavelet Tour of Signal Processing, Academic Press, 1999. 
[32]    Martin D., O'neill M., Hubbard S., and Palmer A. “The Role of Emotion in Explaining Consumer Satisfaction and Future Behavioral Intention,” Journal of Services Marketing, vol. 22, no. 3, pp. 224-236, 2008. DOI:10.1108/08876040810871183
[33]    Mlakar U., Fister I., Brest J., and Potočnik B., “Multi-Objective Differential Evolution for Feature Selection in Facial Expression Recognition Systems,” Expert Systems with Applications, vol. 89, pp. 129-137, 2017. https://doi.org/10.1016/j.eswa.2017.07.037
[34]    Moeini A., Faez K., Sadeghi H., and Moeini H., “2D Facial Expression Recognition Via 3D Reconstruction and Feature Fusion,” Journal of Visual Communication and Image Representation, vol. 35, pp. 1-14, 2016. https://doi.org/10.1016/j.jvcir.2015.11.006
[35]    Mohammadi M., Fatemizadeh E., and Mahoor M., “PCA-based Dictionary Building for Accurate Facial Expression Recognition Via Sparse Representation,” Journal of Visual Communication and Image Representation, vol. 25, no. 5, pp. 1082-1092, 2014. https://doi.org/10.1016/j.jvcir.2014.03.006
[36]    Moore S. and Bowden R., “Local Binary Patterns for Multi-View Facial Expression Recognition,” Computer Vision and Image Understanding, vol. 115, no. 4, pp. 541-558, 2011. https://doi.org/10.1016/j.cviu.2010.12.001
[37]    Nazir M., Jan Z., and Sajjad M., “Facial Expression Recognition Using Histogram of Oriented Gradients Based Transformed Features,” Cluster Computing, vol. 21, pp. 539-548, 2018. https://doi.org/10.1007/s10586-017-0921-5
[38]    Ojala T., Pietikainen M., and Maenpaa T., “Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, 2002. doi: 10.1109/TPAMI.2002.1017623
[39]    Paharia N., Jadon R., and Gupta S., “Feature Selection Using Improved Multiobjective and Opposition-based Competitive Binary Gray Wolf Optimizer for Facial Expression Recognition,” Journal of Electronic Imaging, vol. 31, no. 3, pp. 033039-033039, 2022. DOI:10.1117/1.JEI.31.3.033039
[40]    Perez-Gomez V., Rios-Figueroa H., Rechy-Ramirez E., Mezura-Montes E., and Marin-Hernandez A., “Feature Selection on 2D and 3D Geometric Features to Improve Facial Expression Recognition,” Sensors, vol. 20, no. 17, pp. 4847, 2020. https://doi.org/10.3390/s20174847

[41]    Sanmay D., “Filters, Wrappers and a Boosting-Based Hybrid for Feature Selection,” in Proceedings of the International Conference on Machine Learning, San Francisco, pp.74-81, 2001.
[42]    Siddiqi M., Alruwaili M., and Ali A., “A Novel Feature Selection Method for Video-based Human Activity Recognition Systems,” IEEE Access, vol. 7, pp. 119593-119602, 2019. doi:10.1109/ACCESS.2019.2936621
[43]    Soyel H., Tekguc U., and Demirel H., “Application of NSGA-II to Feature Selection for Facial Expression Recognition,” Computers and Electrical Engineering, vol. 37, no. 6, pp. 1232-1240, 2011. https://doi.org/10.1016/j.compeleceng.2011.01.010
[44]    Story M. and Congalton R., “Accuracy Assessment: A User’s Perspective,” Photogrammetric Engineering and Remote Sensing, vol. 52, no. 3, pp. 397-399, 1986.
[45]    Suto J., Oniga S., and Sitar P., “Comparison of Wrapper and Filter Feature Selection Algorithms on Human Activity Recognition,” in Proceedings of the 6th International Conference on Computers Communications and Control, Oradea, pp. 124-129, 2016. doi: 10.1109/ICCCC.2016.7496749
[46]    Xiong L., Zhao Z., Pan J., Yang H., and Wang W., “Recognition of Heart Sound Based on Wavelet Cepstrum Coefficient and Probabilistic Neural Network,” Hans Journal of Biomedicine, vol. 9, no. 1, pp. 10-16, 2019. DOI:10.12677/HJBM.2019.91002
[47]    Ye Z., Mohamadian H., and Ye Y., “Information Measures for Biometric Identification Via 2D Discrete Wavelet Transform,” in Proceedings of the IEEE International Conference on Automation Science and Engineering, Scottsdale, pp. 835-840, 2007. doi: 10.1109/COASE.2007.4341670
[48]    Young S., Kershaw D., Odell J., and Ollason D., The HTK Book, Cambridge: Entropic Ltd, 1999.
[49]    Youssif A. and Asker W., “Automatic Facial Expression Recognition System Based on Geometric and Appearance Features,” Computer and Information Science, vol. 4, no. 2, pp. 115-124, 2011. DOI:10.5539/cis.v4n2p115
[50]    Zhang D., Fundamentals of Image Data Mining, Springer, 2019. https://doi.org/10.1007/978-3-030-17989-2_3
[51]    Zhao X. and Zhang S., “A Review on Facial Expression Recognition: Feature Extraction and Classification,” IETE Technical Review, vol. 33, no. 5, pp. 505-517, 2016. DOI:10.1080/02564602.2015.1117403