The International Arab Journal of Information Technology (IAJIT)


Human Facial Emotion Recognition using Deep Neural Networks

Humans experience a plethora of different, confusing and nearly indiscernible emotions. Interpreting and understanding these emotions is a rewarding challenge as they play an important role in developing and maintaining interpersonal relationships. Hence, extracting and understanding emotions is paramount to the interaction and communication between human and machine. This understanding aids applications such as brand humanization, sales, advertising and marketing by helping to gauge the response of existing or prospective clients, medical industry, e-learning, law enforcement, automatic counseling system, drunk-driving detection, pain or stress detection, brand value analysis from consumer reactions and the entertainment industry, social interactions and facilitate rational decision making and perception. The recognition of facial emotion is quite difficult problem since there will be a variation of same emotion of single person owing to occlusion, illumination, aging, pose, gender etc. This paper proposes a deep learning model to detect the seven basic emotion classes. The dataset used are CK+, JAFFE and it achieves higher accuracy when compared to the existing approaches.


[1] Abate A., Barra P., Pero C., and Tucci M., “Head Pose Estimation by Regression Algorithm,” Pattern Recognition Letters, vol. 140, pp. 179- 185, 2020.

[2] Aljoloud A., Ullah H., Alanazi A., “Facial Emotion Recognition Using Neighborhood Features,” International Journal of Advanced Computer Science and Applications, Madison, pp. 299-306, 2020.

[3] Bartlett M., Littlewort G., Fasel I., and Movellan J., “Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop, Madison, 2003.

[4] Chen Y. and Hu H., “Facial Expression Recognition by Inter-Class Relational Learning,” IEEE Access, vol. 7, pp. 94106-94117, 2019.

[5] Dalal N. and Triggs B., “Histograms of Oriented Gradients for Human Detection,” in Proceedings of the Computer Vision and Pattern Recognition, San Diego, pp. 886-893, 2005.

[6] Farajzadeh N., Pan G., and Wu Z., “Facial Expression Recognition Based on Meta Probability Codes,” Pattern Analysis and Applications, vol. 17, no. 4, pp. 763-781, 2014.

[7] Georgescu M., Ionescu R., and Popescu M., “Local Learning with Deep and Hand- Crafted Features for Facial Expression Recognition,” IEEE Access, vol. 7, pp. 64827- 64836, 2019.

[8] Goyani M. and Patel N., “Multi-Level Haar Wavelet based Facial Expression Recognition using Logistic Regression,” Indian Journal of Science and Technology, vol. 10, no. 9, pp. 1-9, 2017.

[9] Happy S. and Routray A., “Automatic Facial Expression Recognition Using Features of Salient Facial Patches,” IEEE Transactions on Affective Computing, vol. 6, no. 1, pp. 1-12, 2015.

[10] Jabid T., Kabir M., and Chae O., “Robust Facial Expression Recognition Based on Local Directional Pattern,” Etri Journal, vol. 32, no. 5, pp. 784-794, 2010.

[11] Jain D., Shamsolmoali P., and Sehdev P., “Extended Deep Neural Network for Facial Emotion Recognition,” Pattern Recognition Letters, vol. 120, pp. 69-74, 2019.

[12] Jain N., Kumar S., Kumar U., Shamsolmoali P., and Zareapoor M., “Hybrid Deep Neural Networks for Face Emotion Recognition,” Pattern Recognition Letters, vol. 115, pp. 101-106, 2018.

[13] Jiang R., Ho A., Cheheb I., Al-Maadeed N., Al- Maadeed S., and Bouridane A., “Emotion Recognition from Scrambled Facial Images via Many Graph Embedding,” Pattern Recognition, vol. 67, pp. 245-251, 2017.

[14] Kahou S., Bouthillier X., Lamblin P., Gülçehre Ç., Michalski V., Konda K., Jean S., Froumenty P., Dauphin Y., Boulanger-Lewandowski N., Ferrari R., Mirza M., Warde-Farley D., Courville A., Vincent P., Memisevic R., Pal C., and Bengio Y., “Emotionets: Multimodal Deep Learning Approaches for Emotion Recognition in Video,” Journal Multimodal User Interfaces, vol. 10, pp. 99-111, 2015.

[15] Krestinskaya O. and James A., “Facial Emotion Recognition Using Min-Max Similarity Classifier,” in Proceedings of the International Conference on Advances in Computing, Communications and Informatics, Udupi, pp. 752-758, 2017.

[16] Kumar Y., Verma S., and Sharma S., “Multi-pose Facial Expression Recognition Using Hybrid Deep Learning Model with Improved Variant of Gravitational Search Algorithm,” The International Arab Journal of Information Technology, vol. 19, no. 2, pp. 281-287, 2022.

[17] Li Y., Zeng J., Shan S., and Chen X., “Occlusion Aware Facial Expression Recognition Using CNN with Attention Mechanism,” IEEE Transactions on Image Processing, vol. 28, no. 5, pp. 2439-2450, 2019.

[18] Liu P., Han S., Meng Z., and Tong Y., “Facial Expression Recognition via a Boosted Deep Belief Network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, pp. 1805-1812, 2014.

[19] Lopes A., De Aguiar E., De Souza A., and Oliveira-Santos T., “Facial Expression Recognition with Convolutional Neural Networks: Coping with Few Data and the Training Sample Order,” Pattern Recognition, vol. 61, pp. 610-628, 2017.

[20] Lucey P., Cohn J., Kanade T., Saragih J., Ambadar Z., and Matthews I., “The Extended Cohn-Kanade Dataset (CK++): A Complete Dataset for Action Unit and Emotion-Specified Expression,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, pp. 94-101, 2010.

[21] Minaee S., Minaei M., and Abdolrashidi A., “Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network,” arXiv:1902.01019v1

[cs.CV], 2019.

[22] Nanda A., Im W., Choi K., and Yan H., “Combined Center Dispersion Loss Function for Human Facial Emotion Recognition using Deep Neural Networks 309 Deep Facial Expression Recognition,” Pattern Recognition Letters, vol. 141, pp. 8-15, 2021.

[23] Nithya S., “Emotion Recognition from Facial Expression using Deep Learning,” International Journal of Engineering and Advanced Technology, vol. 8, no. 6S, pp. 91-95, 2019.

[24] Niu B., Gao Z., and Guo B., “Facial Expression Recognition with LBP and ORB Features,” Computational Intelligence and Neuroscience, vol. 2021, pp. 1-10, 2021.

[25] Shah J., Sharif M., Raza M., Murtaza M., and Ur- Rehman S., “Robust Face Recognition Technique under Varying Illumination,” Journal of Applied Research and Technology, vol. 13, no. 1, pp. 97-105, 2015.

[26] Sun A., Li Y., Huang Y., Li Q., and Lu G., “Facial Expression Recognition Using Optimized Active Regions,” Human-Centric Computing and Information Sciences, vol. 8, no. 1, pp. 1-24, 2018.

[27] Wang X., Wang X., and Ni Y., “Unsupervised Domain Adaptation for Facial Expression Recognition Using Generative Adversarial Networks,” Computational Intelligence and Neuroscience, vol. 2018, no. ID 7208794, 2018.

[28] Zhou Y., Xue H., and Geng X., “Emotion Distribution Recognition from Facial Expressions,” in Proceedings of the 23rd ACM International Conference on Multimedia, New York, pp. 1247-1250, 2015.

[29] Zeng N, Zhang H., Song B., Liu W., Li Y., and Dobaie A., “Facial Expression Recognition via Learning Deep Sparse Autoencoders,” Neurocomputing, vol. 273, pp. 643-649, 2018.

[30] Zhang T., Zheng W., Cui Z., Zong Y., and Li Y., “Spatial-Temporal Recurrent Neural Network for Emotion Recognition,” IEEE Transactions on Cybernetics, vol. 49, no. 3, pp.839-847, 2018.

[31] ZhuY., De Silva L., and Ko C., “Using Moment Invariants and Hmm in Facial Expression Recognition,” Pattern Recognition Letters, vol. 23, no. 1, pp. 83-91, 2002.