Abstract The recognition of human facial expressions with the variation of poses is one of the challenging tasks in real-time applications such as human physiological interaction detection, intention analysis, marketing interest evaluation, mental disease diagnosis, etc. This research work addresses the problem of expression recognition from different facial poses at the yaw angle. The major contribution of the paper is the proposal of an autonomous pose variant facial expression recognition framework using the amalgamation of a hybrid deep learning model with an improved quantum inspired gravitational search algorithm. The hybrid deep learning model is the integration of the convolutional neural network and recurrent neural network. The applicability of the hybrid deep learning model can be considered as significant if the feature set is efficiently optimized to have the discriminative features respective to each expression class. Here, the Improved Quantum Inspired Gravitational Search Algorithm (IQI-GSA) is utilized for the selection and optimization of features. The IQI-GSA method is significant for optimizing the features compared to quantum-behaved binary gravitation search algorithm for handing the local optima and stochastic characteristics. Comparing with state-of-art techniques, the proposed framework exhibits the outperformed recognition rate for experimentation on Karolinska Directed Emotional Faces (KDEF) and Japanese Female Facial Expression (JAFFE) datasets.

References

[1] Alexandre G., Soares J., and Thé G., “Systematic Review of 3D Facial Expression Recognition Methods,” Pattern Recognition, vol. 100, no. 3, pp. 107108, 2020.

[2] Barman A. and Dutta P., “Facial Expression Recognition Using Distance and Shape Signature Features,” Pattern Recognition Letters, vol. 145, pp. 254-261, 2021.

[3] Chakraborti T., Chatterjee A., Halder A., and Konar A., “Automated Emotion Recognition Employing a Novel Modified Binary Quantum‐Behaved Gravitational Search Algorithm with Differential Mutation,” Expert Systems, vol. 32, no. 4, pp. 522-530, 2015.

[4] Jain N., Kumar S., Kumar A., Shamsolmoali P., and Zareapoor M., “Hybrid Deep Neural Networks for Face Emotion Recognition,” Pattern Recognition Letters, vol. 115, pp. 101- 106, 2018.

[5] Jeni L., Hashimoto H., and Kubota T., “Robust Facial Expression Recognition Using Near Infrared Cameras,” Journal of Advanced Computational Intelligence and Intelligent Informatics, vol. 16, no. 2, pp. 341-348, 2012.

[6] Kumar Y., Verma S., and Sharma S., “Multi- Pose Facial Expression Recognition using Appearance-based Facial Features,” International Journal of Intelligent Information and Database Systems, vol.13, no. 2-4, pp.172-190, 2020.

[7] Kumar Y., Verma S., and Sharma S., “Quantum- Inspired Binary Gravitational Search Algorithm to Recognize the Facial Expressions,” International Journal of Modern Physics C, vol. 31, no. 10, pp. 2050138, 2020.

[8] Li S. and Deng W., “Deep Facial Expression Recognition: A Survey,” IEEE Transactions on Affective Computing, 2020.

[9] Littlewort G., Bartlett M., Fasel I., Susskind J., and Movellan J., “Dynamics of Facial Expression Extracted Automatically from Video,” in Proceedings of Conference on Computer Vision and Pattern Recognition Workshop, Washington, 2006.

[10] Liu W., Wang Z., Liu X., Zeng N., Liu Y., and Alsaadi F., “A Survey of Deep Neural Network Architectures and Their Applications,” Neurocomputing, vol. 234, pp. 11-26, 2017.

[11] Lundqvist D., Flykt A., and Öhman A., “The Karolinska Directed Emotional Faces (KDEF),” CD ROM from Department of Clinical Neuroscience, Psychology Section, Karolinska Institutet, vol. 91, pp. 630-639, 1998.

[12] Lyons M., Akamatsu S., Kamachi M., and Gyoba J., “Coding Facial Expressions with Gabor Wavelets,” in Proceedings of 3rd IEEE International Conference on Automatic Face and Gesture Recognition, Nara, pp. 200-205, 1998.

[13] Mehrabian A., Communication without Words, Routledge, 2008.

[14] Moghadam M., Nezamabadi-Pour H., and Farsangi M., “A Quantum Behaved Gravitational Search Algorithm,” Intelligent Information Management, vol. 4, no. 6, pp. 390-395, 2012.

[15] Ojala T., Pietikäinen M., and Harwood D., “A Comparative Study of Texture Measures with Classification based on Featured Distributions,” Pattern Recognition, vol. 29, no. 1, pp. 51-59, 1996.

[16] Puthanidam R. and Moh T., “A Hybrid Approach for Facial Expression Recognition,” in Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication, New York, pp. 1-8, 2018.

[17] Rashedi E., Nezamabadi-Pour H., and Saryazdi S., “GSA: A Gravitational Search Algorithm,” Information Sciences, vol. 179, no. 13, pp. 2232- 2248, 2009.

[18] Revina I. and Emmanuel W., “A Survey on Human Face Expression Recognition Techniques,” Journal of King Saud University- Computer and Information Sciences, vol. 33, no. 6, pp. 619-628, 2021.

[19] Soleimanpour-Moghadam M. and Nezamabadi- Pour H., “An Improved Quantum Behaved Gravitational Search Algorithm,” in Proceedings of the 20th Iranian Conference on Electrical Engineering, Tehran, pp. 711-715, 2012.

[20] Sun Z., Hu Z., Chiong R., Wang M., and He W., “Combining the Kernel Collaboration Representation and Deep Subspace Learning for Facial Expression Recognition,” Journal of Circuits, Systems and Computers, vol. 27, no. 08, pp. 1850121, 2018.

[21] Viola P. and Jones M., “Rapid Object Detection using a Boosted Cascade of Simple Features,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, pp. 511-518, 2001.

[22] Wang H., Lei Z., Zhang X., Zhou B., and Peng J., “A Review of Deep Learning for Renewable Energy Forecasting,” Energy Conversion and Management, vol. 198, pp. 111799, 2019.

[23] Zhou L., Shao X., and Mao Q., “A Survey of Micro-Expression Recognition,” Image and Vision Computing, vol. 105, pp. 104043, 2021.

[24] Zhou W. and Jia J., “Training Convolutional Neural Network for Sketch Recognition on Large-Scale Dataset,” The International Arab Journal of Information Technology, vol. 17, no. 1, pp. 82-89, 2020. Multi-Pose Facial Expression Recognition Using Hybrid Deep Learning Model ... 287

[25] Zouari R., Boubaker H., and Kherallah M., “RNN-LSTM Based Beta-Elliptic Model for Online Handwriting Script Identification,” The International Arab Journal of Information Technology, vol. 15, no. 3A, pp. 532-539, 2018. Yogesh Kumar is working as an Assistant Professor at GD Goenka University, Gurugram, Haryana, India. He is pursuing Ph.D in Computer Science and Engineering from UTU, Dehradun with a focus on Human Computer Interface. He is member of International Association of Engineers (IAENG). Shashi Kant Verma is an Associate Professor at G B Pant Engineering College, Pauri- Garhwal, Uttarakhand. He received the Ph. D. from Uttarakhand Technical University, Dehradun U.K. India. He has vast experience as an academician for over 10 years. His area of interest is in Computer Architecture, Microprocessors, and Signal Processing. Sandeep Sharma is a Senior Member of IEEE. He completed his Ph D and Masters in Electronics from University of Delhi and has over 20 years of experience in Research, Industry and Academia. He is currently working as a Head, Academics, at iNurture Education Solutions Pvt Ltd, India. He has published several National and International Journals including many SCI Journals.