In this study, we developed a new system for online Arabic handwriting recognition based on temporal residual networks with multi-head attention model. The main idea behind the application of attention mechanism was to focus on the most relevant parts of the data by a weighted combination of all input sequences. Moreover, we applied beta elliptic approach to represent both kinematic and geometric aspects of the handwriting motion. This approach consists of representing the neuromuscular impulses involving during the writing act. In the dynamic profile, the curvilinear velocity can be fitted by an algebraic sum of overlapped beta functions, while the original trajectory can be rebuilt by elliptic arcs delimited between successive extremum velocity instants. The experiments were conducted on LMCA database containing the trajectory coordinates of 23141 Arabic handwriting letters, and showed very promising results that achieved the recognition rate of 97,12%.

[1] Abdelaziz I., Abdou S., and Al-Barhamtoshy H., “A Large Vocabulary System for Arabic Online Handwriting Recognition,” Pattern Analysis and Applications, vol. 19, no. 4, pp. 1129-1141, 2016. https://doi.org/10.48550/arXiv.1410.4688

[2] Abuzaraida M., Elmehrek M., and Elsomadi E., “Online Handwriting Arabic Recognition System Using K-Nearest Neighbors Classifier and DCT Features,” International Journal of Electrical and Computer Engineering, vol. 11, no. 4, pp. 3584- 3592, 2021. http://doi.org/10.11591/ijece.v11i4.pp3584-3592

[3] Ahsan M., Mahmud M., Saha P., Gupta K., and Siddique Z., “Effect of Data Scaling Methods on Machine Learning Algorithms and Model Performance,” Technologies, vol. 9, no. 3, pp. 52- 59, 2021. https://doi.org/10.3390/technologies9030052

[4] Alimi A., “An Evolutionary Neuro-Fuzzy Approach to Recognize On-Line Arabic Handwriting,” in Proceedings of the 4th International Conference on Document Analysis and Recognition, Ulm, pp. 382-386, 1997. 10.1109/ICDAR.1997.619875

[5] Altwaijry N. and Al-Turaiki I., “Arabic Handwriting Recognition System Using Convolutional Neural Network,” Neural Computing and Applications, vol. 33, no. 7, pp. 2249-2261, 2021.

[6] Alwajih F., Badr E., and Abdou S., “Transformer- based Models for Arabic Online Handwriting Recognition,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 5, pp. 898-905, 2022. 10.14569/IJACSA.2022.01305102

[7] Amudha J., Thakur M., Shrivastava A., Gupta S., Gupta D., and Sharma K., “Wild OCR: Deep Learning Architecture for Text Recognition in Images,” in Proceedings of International Conference on Computing and Communication Networks, Kanpur, pp. 499-506, 2022. DOI:10.1109/ICCCNT45670.2019.8944593

[8] Bezine H., Alimi A., and Sherkat N., “Generation and Analysis of Handwriting Script with the Beta- Elliptic Model,” in Proceedings of 9th International Workshop on Frontiers in Handwriting Recognition, Kokubunji, pp. 515- 520, 2004. 10.1109/IWFHR.2004.45

[9] Boubaker H., Chaabouni A., Tagougui N., Kherallah M., and Alimi A., “Handwriting and Hand Drawing Velocity Modeling by Superposing Beta Impulses and Continuous Training Component,” International Journal of Computer Science Issues, vol. 10, no. 5, pp. 57- 63, 2013.

[10] Boubaker H., Elbaati A., Tagougui N., El Abed H., Kherallah M., and Alimi A., Guide to OCR for Arabic Scripts, Springer, 2012.

[11] Boubaker H., Rezzoug N., Kherallah M., Gorce P., and Alimi A., “Spatiotemporal Representation of 3D Hand Trajectory Based on Beta-Elliptic Models,” Computer Methods in Biomechanics and Biomedical Engineering, vol. 18, no. 15, pp. 1632-1647, 2015. doi: 10.1080/10255842.2014.940331

[12] Chikmagalur A., Devaraj S., Hebbakavadi P., Rajasekhar K., and Reddy U., “Off-Line Signature Confirmation based on Cluster Representations of Geometrical and Statistical Features through Vector Distance, Neural Network and Support Vector Machine Classifiers,” The International Arab Journal on Information Technology, vol. 19, no. 4, pp. 667- 673, 2022. https://doi.org/10.1016/j.patrec.2005.04.013

[13] Dhieb T., Boubaker H., Njah S., Ben Ayed M., and Alimi A., “A Novel Biometric System for Signature Verification Based on Score Level Fusion Approach,” Multimedia Tools and Applications, vol. 81, no. 6, pp. 7817-7845, 2022. https://doi.org/10.1007/s11042-022- 12140-7

[14] Diaz M., Moetesum M., Siddiqi I., and Vessio G., “Sequence-based Dynamic Handwriting Analysis for Parkinson’s Disease Detection with One-Dimensional Convolutions and BiGRUs,” Expert Systems with Applications, vol. 168, pp. 1-34, 2021. https://doi.org/10.1016/j.eswa.2020.114405

[15] Eltay M., Zidouri A., and Ahmad I., “Exploring Deep Learning Approaches to Recognize Handwritten Arabic Texts,” IEEE Access, vol. 8, pp. 89882-89898, 2020. DOI: 10.1109/ACCESS.2020.2994248

[16] Hamdi Y., Boubaker H., and Alimi A., “Online Arabic Handwriting Recognition Using Graphemes Segmentation and Deep Learning Recurrent Neural Networks,” in Proceedings of the Enabling Machine Learning Applications in Data Science, Springer, pp. 281-297, 2021. DOI:10.1007/978-981-33-6129-4_20

[17] Hamdi Y., Boubaker H., Dhieb T., Elbaati A., and Alimi A., “Hybrid DBLSTM-SVM based Beta-Elliptic-CNN Models for online Arabic Characters Recognition,” in Proceedings of the International Conference on Document Analysis and Recognition, Sydney, pp. 545-550, 2019. DOI: 10.1109/ICDAR.2019.00093

[18] He K., Zhang X., Ren S., and Sun J., “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016. 10.1109/CVPR.2016.90 476 The International Arab Journal of Information Technology, Vol. 20, No. 3A, Special Issue 2023

[19] Othmen D., Zouari R., Boubaker H., and Kherallah M., “Temporal Convolution based Skip Connections for Online Arabic Handwriting Recognition,” in Proceedings of the International Arab Conference on Information Technology, Abu Dhabi, pp. 1-5, 2022. DOI: 10.1109/ACIT57182.2022.9994095

[20] Plamondon R., Alimi A., Yergeau P., Leclerc F., “Modelling Velocity Profiles of Rapid Movements: A Comparative Study,” Biological cybernetics, vol. 69, no. 2, pp. 119-128, 1993. DOI: 10.1007/BF00226195

[21] Plamondon R., Guerfali W., The Generation of Handwriting With Delta-Lognormal Synergies, Biological Cybernetics, vol. 78, no. 2, pp. 119- 132, 1998. DOI:10.1007/s004220050419

[22] Sankara Babu B., Nalajala S., Sarada K., Muniraju Naidu, V., Yamsani, N., and Saikumar K., “Machine Learning Based Online Handwritten Telugu Letters Recognition for Different Domains,” in Proceedings of the A Fusion of Artificial Intelligence and Internet of Things for Emerging Cyber Systems, pp. 227-241, Springer, 2021. DOI:10.1007/978-3-030-76653-5_12

[23] Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A., Kaiser L., and Polosukhin I., “Attention is All You Need,” in Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, pp. 1-11, 2017.

[24] Wilson-Nunn D., Lyons T., Papavasiliou A., and Ni H., “A Path Signature Approach to Online Arabic Handwriting Recognition,” in Proceedings of the 2nd International Workshop on Arabic and Derived Script Analysis and Recognition, London, London, pp. 135-139, 2018. DOI: 10.1109/ASAR.2018.8480300

[25] Zouari R., Boubaker H., and Kherallah M., “A Time Delay Neural Network for Online Arabic Handwriting Recognition,” in Proceedings of the International Conference on Intelligent Systems Design and Applications, Portugal, pp. 1005- 1014, 2016. DOI:10.1007/978-3-319-53480-0_99

[26] Zouari R., Boubaker H., and Kherallah M., “RNN-LSTM based Beta-Elliptic Model for Online Handwriting Script Identification,” The International Arab Journal on Information Technology, vol. 15, no. 3A, pp. 532-539, 2018.

[27] Zouari R., Boubaker H., and Kherallah M., “Two Staged Fuzzy SVM Algorithm and Beta-Elliptic Model for Online Arabic Handwriting Recognition,” in Proceedings of the International Conference on Artificial Neural Networks, Alghero, pp. 450-458, 2017. DOI:10.1007/978-3- 319-68612-7_51

[28] Zouari R., Boubaker H., and Kherallah M., “Hybrid TDNN-SVM Algorithm for Online Arabic Handwriting Recognition,” in Proceedings of the 16th International Conference on Hybrid Intelligent Systems , Marrakech, pp. 113-123, 2016. DOI:10.1007/978-3-319-52941-7_12

[29] Zouari R., Boubaker H., and Kherallah M.., “Towards Online Handwriting Recognition System Based on Reinforcement Learning Theory,” in Proceedings of the International Conference on Neural Information Processing, Bangkok, pp. 562-570, 2020.