Abstract Recurrent Neural Network (RNN) has achieved the state-of-the-art performance in a wide range of applications dealing with sequential input data. In this context, the proposed system aims to classify the online handwriting scripts based on their labelled pseudo-words. To avoid the vanishing gradient problem, we have used a variant of recurrent network with Long Short-Term Memory. The representation of the sequential aspect of the data is done through the beta-elliptic model. It allows extracting the dynamics and kinematics profiles of different strokes constituting a script over the time. This system was assessed with a large vocabulary containing scripts from ADAB, UNIPEN and PENDIGIT databases. The experiments results show the effectiveness of the proposed system which reached a high recognition rate with only one recurrent layer and using the dropout technique.

References

[1] Alimi A., Beta Neuro-Fuzzy Systems, TASK Quarterly Journal, Special Issue on Neural Networks, vol. 7, no.1, pp. 23-41, 2003.

[2] Assawinjaipetch P., Sornlertlamvanich V., Shirai K., and Marukata S., Recurrent Neural Network with Word Embedding for Complaint Classification, in Proceedings of the 3th International Workshop on Worldwide Language Service Infrastructure and Second Workshop on Open Infrastructures and Analysis Frameworks for Human Language Technologies, Osaka, pp. 36- 43, 2016.

[3] Chaabouni A., Boubaker H., Kherallah M., El Abed H., and Alimi A., Static and Dynamic Features for Writer Identification Based on Multi- Fractals, The International Arab Journal on Information and Technology, vol. 11, no. 4, pp. 416-424, 2014.

[4] Dhieb T., Ouarda W., Boubaker H., Halima M., and Alimi A., Online Arabic Writer Identification Based on Beta-Elliptic Model, in Proceedings of the 15th International Conference on Intelligent Systems Design and Applications, Marrakech, pp. 74-79, 2015.

[5] Graves A., Mohamed A., and Hinton G., Speech Recognition with Deep Recurrent Neural Networks, in Proceedings of the International Conference on Acoustics, Speech and Signal Processing, pp. 6645-6649, 2013

[6] Haboubi S., Maddouri S., and Amiri H., Discrimination between Arabic and Latin from Bilingual Documents, in Proceedings of the International Conference on Communications, Computing and Control Applications, Hammamet, pp. 1-6, 2011.

[7] Hasan A., Afzal M., Shafait F., Liwicki M., and Breuel T., A Sequence Learning Approach for Multiple Script Identification, in Proceedings of the 13th International Conference on Document Analysis and Recognition, Tunis, pp. 1046-1050, 2015.

[8] Hochreiter S. and Schmidhuber J., Long short- Term Memory, Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.

[9] Jlaiel M., Kanoun S., Alimi A., and Mullot R., Three Decision Levels Strategy for Arabic and Latin Texts Differentiation in Printed and Handwritten Natures, in Proceedings of the 9th International Conference on Document Analysis and Recognition, pp. 1103-1107, 2007.

[10] Kherallah M., Bouri F., and Alimi A., On-Line Arabic Handwriting Recognition System Based on Visual Encoding and Genetic Algorithm, Engineering Applications of Artificial Intelligence, vol. 22, no. 1, pp. 153-170, 2009.

[11] Kherallah M., Haddad L., Alimi A., and Mitiche A., On-Line Handwritten Digit Recognition Based on Trajectory and Velocity Modeling, Pattern Recognition Letters, vol. 29, no. 5, pp. 580-594, 2008.

[12] Kulkarni A., Upparamani P., Kadkol R., and Tergundi P., Script Identification from Multilingual Text Documents, International Journal of Advanced Research in Computer and Communication Engineering, vol. 4, no. 6, pp. 15-19, 2015.

[13] MacQueen J., Some Methods for Classification and Analysis of Multivariate Observations, in Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, no. 14, pp. 281-297, 1967.

[14] Mesnil G., He X., Deng L., and Bengio Y., Investigation of Recurrent-Neural-Network Architectures and Learning Methods for Spoken Language Understanding, in Proceedings of the 14th Annual Conference of the International Speech Communication Association, Lyon, pp. 3771-3775, 2013.

[15] Mezghani A., Slimane F., Kanoun S., and M rgner V., Identification of Arabic/French Handwritten/Printed Words using GMM-Based System, in Proceedings of the 11th French Information Retrieval Conference, Nancy, pp. 371-374, 2014.

[16] Moon T., Choi H., Lee H., and Song I., RNNDROP: A Novel Dropout for RNNS in ASR, in Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding, Scottsdale, pp. 65-70, 2015.

[17] Namboodiri A. and Jain A., Online Handwritten Script Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 124-130, 2004.

[18] Rouhou A., Abdelhedi Z., and Kessentini Y., A HMM-Based Arabic/Latin Handwritten/Printed Identification System, in Proceedings of the RNN-LSTM Based Beta-Elliptic Model for Online Handwriting Script Identification 539 International Conference on Hybrid Intelligent Systems, Seoul, pp. 298-307, 2016.

[19] Sa dani A. and Echi A., Pyramid Histogram of Oriented Gradient for Machine- Printed/Handwritten and Arabic/Latin Word Discrimination, in Proceedings of the 6th International Conference of Soft Computing and Pattern Recognition, Tunis, pp. 267-272, 2014.

[20] Sa dani A., Echi A., and Belaid A., Identification of Machine-Printed and Handwritten Words in Arabic and Latin Scripts, in Proceedings of the 12th International Conference on Document Analysis and Recognition, Washington, pp. 798- 802, 2013.

[21] Sa daniA., Kacem A., and Belaid A., Co- Occurrence Matrix of Oriented Gradients for Word Script And Nature Identification, in Proceedings of the 13th International Conference on Document Analysis and Recognition, Tunis, pp. 16-20, 2015.

[22] Sak H., Senior A., and Beaufays F., Long Short- Term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling, in Proceedings of the 15th Annual Conference of the International Speech Communication Association, Singapore, pp. 1102-1128, 2014.

[23] Singh A. and Lawahar C., Can RNNs Reliably Separate Script and Language at Word and Line Level, in Proceedings of the 13th International Conference on Document Analysis and Recognition, Tunis, pp. 976-980, 2015.

[24] Singh P., Sarkar R., Nasipuri M., and Doermann D., Word-Level Script Identification for Handwritten Indic Scripts, in Proceedings of the 3rd International Conference on Eco-friendly Computing and Communication Systems, Mangalore, pp. 1106-1110, 2015.

[25] Tan G., Viard-Gaudin C., and Kot A., Information Retrieval Model for Online Handwritten Script Identification, in Proceedings of the 10th International Conference on Document Analysis and Recognition, Barcelona, pp. 336-340, 2009.

[26] Zouari R., Boubaker H., and Kherallah M., Hybrid TDNN-SVM Algorithm for Online Arabic Handwriting Recognition, in Proceedings of the 16th International Conference on Hybrid Intelligent Systems, Seoul, pp. 113-123, 2016.

[27] Zouari R., Boubaker H., and Kherallah M., A Time Delay Neural Network for Online Arabic Handwriting Recognition, in Proceedings of the International Conference on Intelligent Systems Design and Applications, Porto, pp. 1005-1014, 2016.

[28] Zouari R., Mokni R., and Kherallah M., Identification and Verification System of Offline Handwritten Signature Using Fractal Approach, in Proceedings of the First International Conference on Image Processing Applications and Systems, Sfax, pp. 1-4, 2014. Ramzi Zouari he is a Ph.D. Student at the National School of Engineering of Sfax, Tunisia. He obtained the master degree in computer science from the University of Sfax in 2012. His main research interest is in the area of online Handwriting recognition and handwriting trajectory modeling. Houcine Boubaker graduated in Electrical Engineering in 1995, obtained a master degree in Systems Analyses and Digital Signal Processing in 1997. He is a researcher in Electrical and Computer Engineering at the University of Sfax and affiliate to the Research Groups in Intelligent Machines laboratory (ReGIM). His research interest includes trajectories modeling and pattern recognition. He focuses his research on drawing, handwriting and arm-hand movements modeling and Analyses. Monji Kherallah received the engineering Diploma degree and the PhD in electrical engineering, in 1989 and 2008, respectively from University of Sfax (ENIS), Tunisia. He is a member in Research Group of Intelligent Machines: REGIM. His research interest includes the following projects: "i- Bag", "i-House" and "i-Car . The techniques used are based on methods intelligent, such as neural network, logic fuzzy, genetic algorithm, etc., He is a reviewer of several international journals.