Abstract This paper represents a system which can understand and react appropriately to human facial expression for nonverbal communications. The considerable events of this system are detection of human emotions, eye blinking, head nodding and shaking. The key step in the system is to appropriately recognize a human face with acceptable labels. This system uses currently developed OpenCV Haar Feature-based Cascade Classifier for face detection because it can detect faces to any angle. Our system can recognize emotion which is divided into several phases: segmentation of facial regions, extraction of facial features and classification of features into emotions. The first phase of processing is to identify facial regions from real time video. The second phase of processing identifies features which can be used as classifiers to recognize facial expressions. Finally, an artificial neural network is used in order to classify the identified features into five basic emotions. It can also detect eye blinking accurately. It works for the active scene where the eye moves freely and the head and the camera moves independently in all directions of the face. Finally, this system can identify the natural head nodding and shaking that can be recognized in real-time using optical flow motion tracking and find the direction of head during the head movement for nonverbal communication.

References

[1] Ahonen T., Hadid A., and Pietik inen M., Face Recognition with Local Binary Patterns, in proceedings of European Conference on Computer Vision, Prague, pp. 469-481, 2004. (18) (19) %100 TNFNFPTP TNTPOverall %100 FNTP TPDetection Real Time Facial Expression Recognition for Nonverbal Communication 287

[2] Bhaskar T., Keat F., Ranganath S., and Venkatesh Y., Blink Detection and Eye Tracking for Eye Localization, in proceedings of Conference on Convergent Technologies for Asia-Pacific Region, Bangalore, pp. 821-824, 2003.

[3] Bouhabba E., Shafie A., and Akmeliawati R., Support Vector Machine for Face Emotion Detection on Real Time Basis, in proceedings of 4th International Conference on Mechatronics, Kuala Lumpur, pp. 1-6, 2011.

[4] Cantelmo N., Rapid Facial Expression Classification Using Artificial Neural Networks, Northwestern University, http://steadystone.com/ research/ml07/, Last Visited 2007.

[5] Cassell J., Nudge nudge wink wink Elements of Face-to-Face Conversation for Embodied Conversational Agents, MIT Press, 2000.

[6] Chau M. and Betke M., Real Time Eye Tracking and Blink Detection with USB Cameras, Boston University Computer Science Technical Report, USA, no. 2005-12, 2005.

[7] Darwin C., The Expression of the Emotions in Man and Animals, Oxford University Press, 1998.

[8] Ekman P. and Friesen W., Constants across Cultures in the Face and Emotion, Personality and Social Psychology, vol. 17, no. 2, pp. 124- 129, 1971.

[9] Givens D., Dictionary of Gestures, Signs and body Language Cues, Center for Nonverbal Studies Press, 2002.

[10] Gorodnichy D., On Importance of Nose for Face Tracking, in Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, Washington, pp. 181-186, 2002.

[11] Gorodnichy D., Second Order Change Detection, and its Application to Blink-controlled Perceptual Interfaces, IASTED Conference on Visualization, Imaging and Image Processing, Benalmadena, pp. 140-145, 2003.

[12] Grauman K., Betke M., Lombardi J., Gips J., and Bradski G., Communication Via Eye Blinks and Eye-brow Rises: Video-based Human-Computer Interfaces, Universal Access in the Information Society, vol. 2, no. 4, pp. 359-373, 2003.

[13] Happy S., George A., and Routray A., A Real Time Facial Expression Classification System using Local Binary Patterns, in Proceedings of 4th International Conference on Intelligent Human Computer Interaction, Kharagpur, pp. 1-5, 2012.

[14] Ito A., Wang X., Suzuki M., and Makino S., Smile and Laughter Recognition Using Speech Processing and Face Recognition from Conversation Video, in Proceedings of International Conference on Cyberworlds, Singapore, pp. 437-444, 2005.

[15] Kawato S. and Ohya J., Real-time Detection of Nodding and Head-shaking by Directly Detecting and Tracking the Between-Eyes, in Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, pp. 40-45, 2000.

[16] Lin D., Facial Expression Classification using PCA and Hierarchical Radial Basis Function Network, Journal of Information Science and Engineering, vol. 22, no. 5, pp. 1033-1046, 2006.

[17] Magee J., Scott M., Waber B., and Betke M., Eyekeys: A Real-Time Vision Interface Based on Gaze Detection from a Low-Grade Video Camera, IEEE Workshop on Real-Time Vision for Human-Computer Interaction, Washington, pp. 159-166, 2004.

[18] Mehrabian A., Communication without Words, Psychology Today, vol. 2, pp. 53-56, 1968.

[19] Moriyama T., Kanade T., Cohn J., Xiao J., Ambadar Z., Gao J., and Imanura M., Automatic Recognition of Eye Blinking in Spontaneously occurring Behaviour, in Proceedings of 16th International Conference on Pattern Recognition, Quebec, pp.78-81, 2002.

[20] Morris D., Bodytalk: The Meaning of Human Gestures, Crown Publishers, 1994.

[21] Morris T., Zaidi F., and Blenkhorn P., Blink Detection for Real-time Eye Tracking, Networking and Computer Applications, vol. 25, no. 2, pp. 129-143, 2002.

[22] Paschero M., Vescovo G., Benucci L., Rizzi A., Santello M., Fabbri G., and Frattale F., A Real Time Classi er for Emotion and Stress Recognition in a Vehicle Driver, in Proceedings of IEEE International Symposium on Industrial Electronics, Hangzhou, pp. 1690- 1695, 2012.

[23] Rabiner L., A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition, IEEE, vol. 77, no. 2, pp. 257-286, 1989.

[24] Rurainsky J. and Eisert P., Eye Center Localization using Adaptive Templates, in Proceedings of the CVPR Workshop on Face Processing in Video, Washington, pp. 67-67, 2004.

[25] Shan C., Gong S., and McOwan P., Facial Expression Recognition Based on Local Binary Patterns: A Comprehensive Study, Image Vision Computing, vol. 27, no. 6, pp. 803-816, 2009.

[26] Shan C., Smile Detection by Boosting Pixel Differences, IEEE Transactions on Image Processing, vol. 21, no. 1, pp. 431-436, 2012.

[27] Shi J. and Tomasi C., Good Features to Track, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, pp. 593-600, 1994. 288 The International Arab Journal of Information Technology, Vol. 15, No. 2, March 2018

[28] Shojaeilangari S., Yun Y., and Khwang T., Person Independent Facial Expression Analysis using Gabor Features and Genetic Algorithm, in Proceedings of 8th International Conference on information Communications and Signal Processing, Singapore, pp.1-5, 2011.

[29] Thai L., Tat p., and Hai T., Facial Expression Classification Based on Multi Artificial Neural Network, in Proceeding of Advance Computing and Applications, Vietnam, pp. 125-133, 2010.

[30] Thai L., Hai T., and Nguyen N., A Facial Expression Classification System Integrating Canny, Principal Component Analysis and Artificial Neural Network, Machine Learning and Computing, vol. 1, no. 4, pp. 388-393, 2011.

[31] Viola P. and Jones M., Rapid Object Detection using a Boosted Cascade of Simple Features, in Proceeding of IEEE Conference Computer Vision Pattern Recognition, Kauai , pp. 511-518, 2001.

[32] Viola P. and Jones M., Robust Real-Time Face Detection, Computer Vision, vol. 57 no. 2, pp. 137-154, 2004.

[33] Wang H., Mohamad D., and Ismail N., An Efficient Parameters Selection for Object Recognition Based Colour Features in Traffic Image Retrieval, The International Arab Journal of Information Technology, vol. 11, no. 3, pp. 308-314, 2014.

[34] Wang Y., Ai H., Wu B., and Huang C., Real Time Facial Expression Recognition with Adaboost, in Proceedings of the Pattern Recognition 17th International Pattern Recognition, Cambridge, pp. 926-929, 2004.

[35] Wei X., Zhu Z., Yin L., and Ji Q., A Real-time Face Tracking and Animation System, CVPR Workshop on Face Processing in Video, Washington, pp. 71-71, 2004.

[36] Zhang Z, Feature-Based Facial Expression Recognition: Sensitivity Analysis and Experiments with a Multi-Layer Perceptron, International Journal of Pattern Recognition and Artificial Intelligence, vol. 13, no. 6, pp. 893-911, 1999.

[37] Zhao-Yi P., Zhi-Qiang W., and Yu Z, Application of Mean Shift Algorithm in Real- time Facial Expression Recognition, in Proceeding of International Symposium on Computer Network and Multimedia Technology, Wuhan, pp. 1-4, 2009. Md. Sazzad Hossain was born in Dhaka, Bangladesh, in 1984. He received the B.Sc. (Engg.) in Computer Science and Engineering from the Mawlana Bhashani Science and Technology University, Tangail, Bangladesh, in 2008, and the M.Sc. (Engg.) in Computer Science and Engineering from the same university in 2014. Now he is a faculty member of Department of Computer Science and Engineering, Mawlana Bhashani Science and Technology University. His current research interests include human computer interaction, embedded system, VLSI design, reconfigurable architecture and cryptography/information security. Mohammad Abu Yousuf received the B.Sc.(Engineering) degree in Computer Science and Engineering from Shahjalal University of Science and Technology, Sylhet, Bangladesh in 1999, the Master of Engineering degree in Biomedical Engineering from Kyung Hee University, South Korea in 2009, and the Ph.D. degree in Science and Engineering from Saitama University, Japan in 2013. In 2003, he joined as a Lecturer in the Department of Computer Science and Engineering, Mawlana Bhashani Science and Technology University, Tangail, Bangladesh. In 2014, he moved to the Jahangirnagar University as an Assistant Professor in the Institute of Information Technology. His research interests include Medical Image Processing, Human- Robot Interaction and Computer Vision.