The International Arab Journal of Information Technology (IAJIT)

..............................
..............................
..............................


Enhanced Long Short-Term Memory (ELSTM) Model for Sentiment Analysis

#
Sentiment analysis is used to embed an extensive collection of reviews and predicts people's opinion towards a particular topic, which is helpful for decision-makers. Machine learning and deep learning are standard techniques, which make the process of sentiment analysis simpler and popular. In this research, deep learning is used to analyze the sentiments of people. It has an ability to perform automatic feature extraction, which provides better performance, a more vibrant appearance, and more reliable results than conventional feature-based techniques. Traditional approaches were based on complicated manual feature extractions that were not able to provide reliable results. Therefore, the presented study aimed to improve the performance of the deep learning approach by combining automatic feature extraction with manual feature extraction techniques. The enhanced ELSTM model is proposed with hyper-parameter tuning in previous Long Short-Term Memory (LSTM) to get better results. Based on the results, a novel model of sentiment analysis and novel algorithm are proposed to set the benchmark in the field of textual classification and to describe the procedure of the developed model


[1] AI-Smadi M., Talafha B., Al-Ayyoub M., and Jararweh Y., “Using Long Short-Term Memory Deep Neural Networks for Aspect-Based Sentiment Analysis of Arabic Reviews,” International Journal of Machine Learning and Cybernetics, vol. 10, no. 8, pp. 2163-2175, 2019.

[2] Aker A., Petrak J., and Sabbah F., “An Extensible Multilingual Open Source Lemmatizer,” in Proceedings of the International Conference Recent Advances in Natural Language Processing, Varna, pp. 40-45, 2017.

[3] An H. and Moon N., “Design of Recommendation System for Tourist Spot Using Sentiment Analysis based on CNN-LSTM,” Journal of Ambient Intelligence and Humanized Computing, pp. 1-11, 2019.

[4] Araque O., Corcuera-Platas I., Sánchez-Rada J., and Iglesias C., “Enhancing Deep Learning Sentiment Analysis with Ensemble Techniques in Social Applications,” Expert Systems with Applications, vol. 77, pp. 236-246, 2017.

[5] Arras L., Montavon G., Müller K., and Samek W., “Explaining Recurrent Neural Network Predictions in Sentiment Analysis,” arXiv preprint arXiv: 1706.07206, 2017.

[6] Bao W., Yue J., and Rao Y., “A Deep Learning Framework for Financial Time Series Using Stacked Autoencoders and Long-Short Term Memory,” PloS One, vol. 12, no. 7, pp. e0180944, 2017.

[7] Baziotis C., Pelekis N., and Doulkeridis C., “Datastories at Semeval-2017 Task 4: Deep Lstm with Attention for Message-Level and Topic- Based Sentiment Analysis,” in Proceedings of the 11th International Workshop on Semantic Evaluation, Vancouver, pp. 747-754, 2017.

[8] Bhati B., Rai C., Balamurugan B., and Al- Turjman F., “An Intrusion Detection Scheme Based on the Ensemble of Discriminant Classifiers,” Computers and Electrical Engineering, vol. 86, pp. 106742, 2020.

[9] Bilgin M. and Köktaş H., “Sentiment Analysis with Term Weighting and Word Vectors,” The International Arab Journal of Information Technology, vol. 16, no. 5, pp. 953-959, 2019.

[10] Cambria E., “Affective Computing and Sentiment Analysis,” IEEE Intelligent Systems, vol. 31, no. 2 102-107, 2016.

[11] Chen W., Jiang M., Zhang W., and Chen Z., “A Novel Graph Convolutional Feature Based Convolutional Neural Network for Stock Trend Prediction,” Information Sciences, vol. 556 pp. 67-94, 2021.

[12] Chong E., Han C., and Park F., “Deep Learning Networks for Stock Market Analysis and Prediction: Methodology, Data Representations, and Case Studies,” Expert Systems with Applications, vol. 83, pp. 187-205, 2017.

[13] Chopra S., Balakrishnan S., and Gopalan R., “Dlid: Deep Learning for Domain Adaptation by Interpolating between Domains,” in the proceedings of the ICML, Workshop on Representation Learning, Atlanta, 2013.

[14] Cliché M., “Bb-Twtr at Semeval-2017 Task 4: Twitter Sentiment Analysis with Cnns and Lstms,” arXiv preprint arXiv: 1704.06125, 2017.

[15] De Clercq O., Lefever E., Jacobs G., Carpels T., and Hoste V., “Towards an Integrated Pipeline for Aspect-Based Sentiment Analysis in Various Domains,” in Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, 854 The International Arab Journal of Information Technology, Vol. 18, No. 6, November 2021 Sentiment and Social Media Analysis, Copenhagen, pp. 136-142, 2017.

[16] Dhandayudam P. and Krishnamurthi I., “Rough Set Approach for Characterizing Customer Behaviour,” Arabian Journal for Science and Engineering, vol. 39, no. 6, pp. 4565-4576, 2014.

[17] Dyer C., Kuncoro A., Ballesteros M., and Smith N., “Recurrent Neural Network Grammars,” arXiv preprint arXiv: 1602.07776, 2016.

[18] Fersini E., Messina E., and Pozzi F., “Sentiment Analysis: Bayesian Ensemble Learning,” Decision Support Systems, vol. 68, pp. 26-38, 2014.

[19] Friedrichs J., Mahata D., and Gupta S., “Infynlp at SMM4H Task 2: Stacked Ensemble of Shallow Convolutional Neural Networks for Identifying Personal Medication Intake from Twitter,” arXiv preprint arXiv: 1803.07718, 2018.

[20] Gabbard S., Yang J., and Liu J., “Quora Insincere Question Classification,” Baskin Engineering, University of California, Santa Cruz, pp. 1- 6, 2018.

[21] Graves A., “Generating Sequences with Recurrent Neural Networks,” arXiv preprint arXiv: 1308.0850, 2013.

[22] Graves A., Mohamed A., and Hinton G., “Speech Recognition with Deep Recurrent Neural Networks,” in Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, pp. 6645-6649, 2013.

[23] Gupta R. and Jivani A., “Analyzing The Stemming Paradigm,” in Proceedings of International Conference on Information and Communication Technology for Intelligent Systems, Ahmedabad, pp. 333-342, 2017.

[24] Heaton J., Polson N., and Witte J., “Deep Learning for Finance: Deep Portfolios,” Applied Stochastic Models in Business and Industry, vol. 33, no. 1, pp. 3-12, 2017.

[25] Huang M., Cao Y., and Dong C., “Modeling Rich Contexts for Sentiment Classification with Lstm,” arXiv preprint arXiv: 1605.01478, 2016.

[26] Kim J., Tur G., Celikyilmaz A., Cao B., and Wang Y., “Intent Detection using Semantically Enriched Word Embeddings,” in Proceedings of IEEE Spoken Language Technology Workshop, San Diego, pp. 414-419, 2016.

[27] Kim Y., “Convolutional Neural Networks for Sentence Classification,” arXiv preprint arXiv: 1408.5882, 2014.

[28] Kraus M. and Feuerriegel S., “Decision Support from Financial Disclosures with Deep Neural Networks and Transfer Learning,” Decision Support Systems, vol. 104, pp. 38-48, 2017.

[29] Lample G., Ballesteros M., Subramanian S., Kawakami K., and Dyer C., “Neural Architectures for Named Entity Recognition,’ arXiv preprint arXiv: 1603.01360, 2016.

[30] LeCun Y., Bengio Y., and Hinton G., “Deep learning,” Nature, vol. 521, pp. 436-444, 2015.

[31] Lee S. and Yoo S., “A New Method for Portfolio Construction using A Deep Predictive Model,” in Proceedings of the 7th International Conference on Emerging Databases, pp. 260-266, 2018.

[32] Li C., Xu B., Wu G., He S., Tian G., and Zhou Y., “Parallel Recursive Deep Model for Sentiment Analysis, in Proceedings of Pacific-Asia Conference on Knowledge Discovery and Data Mining, Ho Chi Minh, pp. 15-26, 2015.

[33] Ma Y., Peng H., Khan T., Cambria E, and Hussain A., “Sentic LSTM: A Hybrid Network for Targeted Aspect-Based Sentiment Analysis,” Cognitive Computation, vol. 10, no. 4, pp. 639-650, 2018.

[34] Nandan M., Khargonekar P., and Talathi S., “Fast SVM Training Using Approximate Extreme Points,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 59-98, 2014.

[35] Ortigosa A., Martín J., and Carro R., “Sentiment Analysis in Facebook and its Application to E- Learning,” Computers in Human Behavior, vol. 31, pp. 527-541, 2014.

[36] Pennington J., Socher R., and Manning C., “Glove: Global Vectors for Word Representation,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, Doha, pp. 1532-1543, 2014.

[37] Poria S., Cambria E., and Gelbukh A., “Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-Level Multimodal Sentiment Analysis,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, Lisbon, pp. 2539- 2544, 2015.

[38] Poria S., Cambria E., Gelbukh A., Bisio F., and Hussain A., “Sentiment Data Flow Analysis By Means Of Dynamic Linguistic Patterns,” IEEE Computational Intelligence Magazine, vol. 10, no. 4, pp. 26-36, 2015.

[39] Poria S., Cambria E., Ku L., Gui C., and Gelbukh A., “A Rule-Based Approach to Aspect Extraction From Product Reviews,” in Proceedings of the Second Workshop on Natural Language Processing for Social Media, Dublin, pp. 28-37, 2014.

[40] Ramos J., “Using Tf-Idf To Determine Word Relevance in Document Queries,” in Proceedings of the 1st Instructional Conference on Machine Learning, pp. 29-48, 2003.

[41] Sangeetha K. and Prabha D., “Sentiment Analysis of Student Feedback Using Multi-Head Attention Fusion Model of Word and Context Embedding for LSTM,” Journal of Ambient Intelligence and Humanized Computing, vol. 12, no. 3, pp. 4117- 4126, 2021. Enhanced Long Short-Term Memory (ELSTM) Model for Sentiment Analysis 855

[42] Santos C. and Gatti M., “Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts,” in Proceedings of COLING, the 25th International Conference on Computational Linguistics: Technical Papers, Dublin, pp. 69-78. 2014.

[43] Sohangir S., Wang D., Pomeranets A., and Khoshgoftaar T., “Big Data: Deep Learning for Financial Sentiment Analysis,” Journal of Big Data, vol. 5, no. 1, pp. 1-25, 2018.

[44] Sosa P., “Twitter Sentiment Analysis Using Combined Lstm-Cnn Models,” Eprint Arxiv, pp. 1-9, 2017.

[45] Tahon M. and Devillers L., “Towards a Small Set of Robust Acoustic Features for Emotion Recognition: Challenges,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 1, pp. 16-28, 2016.

[46] Tiwari D. and Singh N., “Ensemble Approach for Twitter Sentiment Analysis,” International Journal of Information Technology and Computer Science, vol. 11, no. 8, pp. 20-26, 2019.

[47] Tiwari D. and Singh N., “Sentiment Analysis of Digital India using Lexicon Approach,” in Proceedings of 6th International Conference on Computing for Sustainable Global Development, New Delhi, pp. 1189-1193, 2019.

[48] Wang S. and Manning C., “Baselines and Bigrams: Simple, Good Sentiment and Topic Classification,” in Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Jeju, pp. 90-94, 2012.

[49] Wang X., Jiang W., and Luo Z., “Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts,” in Proceedings of COLING, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, pp. 2428-2437, 2016.

[50] Wen S., Wei H., Yang Y., Guo Z., Zeng Z., Huang T., and Chen Y., “Memristive LSTM Network for Sentiment Analysis,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 3, pp. 1794-1804, 2019.

[51] Xu L., Wang J., Li X., Cai F., Tao Y., and Gulliver T., “Performance Analysis and Prediction for Mobile Internet of Things (IoT) Networks: A CNN Approach,” IEEE Internet of Things Journal, vol. 8, no. 17, pp. 13355-13366, 2021. Dimple Tiwari is a PHD scholar in Computer Science and Engineering Department at Ambedkar Institute of Advanced Communication Technologies and Research– AIACTR, Delhi, India. She is also a Microsoft certified in .Net framework. Her areas of interest include Sentiment Analysis, Artificial Intelligence, Information Security, Internet of things (IoT), Big Data. She has published various research papers in reputed International Journals/Conferences and contributed Book Chapters. Bharti Nagpal is currently working as Assistant Professor in Computer Science and Engineering Department at Ambedkar Institute of Advanced Communications Technologies and Research–AIACTR, Delhi, India. She has 21 years of teaching experience. Her areas of interest include Sentiment Analysis, Artificial Intelligence, Information Security, Data mining and Data Warehouse, Internet of things (IoT), Big Data. She has published various research papers in reputed International Journals/Conferences and contributed Book Chapters.