Recent advancements in Natural Language Processing (NLP) have enabled efficient and accurate sentiment analysis through pre-trained language models. This study proposes a lightweight framework leveraging the Distilled Robustly Optimized BERT Approach (DistilRoBERTa) architecture to analyze public sentiment on climate change across twitter from 2011 to 2022. Unlike prior work, our approach integrates multi-domain datasets (International Survey on Emotion Antecedents and Reactions (ISEAR), Multimodal EmotionLines Dataset (MELD), GoEmotions) to fine-tune the model for multi-class emotion recognition, capturing nuanced categories such as fear, anger, and optimism. We conduct a systematic comparison of transformer-based models (Bidirectional Encoder Representations from Transformers (BERT), A Lite BERT (ALBERT), DistilRoBERTa) and traditional deep learning architectures (Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), demonstrating that DistilRoBERTa achieving comparable accuracy (95.9% on Internet Movie Database (IMDB)) with 6× faster inference than RoBERTa. The framework integrates multi-domain datasets such as ISEAR, MELD, and GoEmotions to enhance emotion recognition coverage across seven climate-relevant categories. Longitudinal analysis of 130,000 tweets reveals a significant sentiment shift from optimism (2011-2018) to pessimism (2019-2022), driven by policy inefficacy. Our framework highlights the scalability of distilled models for real-time social media analytics and provides a computational blueprint for scalable policy analytics, enabling real-time integration of NLP into sustainability governance frameworks.

[1] Ahamad N. and Ariffin M., “Assessment of Knowledge, Attitude and Practice Towards Sustainable Consumption among University Students in Selangor, Malaysia,” Sustainable Production and Consumption, vol. 16, pp. 88-98, 2018. https://doi.org/10.1016/j.spc.2018.06.006

[2] Almulhim A., “Understanding Public Awareness and Attitudes Toward Renewable Energy Resources in Saudi Arabia,” Renewable Energy, vol. 192, pp. 572-582, 2022. https://doi.org/10.1016/j.renene.2022.04.122

[3] Aqlan W., Ali G., Rajab K., and et al., “Thalassemia Screening by Sentiment Analysis on Social Media Platform Twitter,” Computers, Materials and Continua, vol. 76, no. 1, pp. 665- 686, 2023. https://doi.org/10.32604/cmc.2023.039228

[4] Araque O., Corcuera-Platas I., Sanchez-Rada J., and Iglesias C., “Enhancing Deep Learning Sentiment Analysis with Ensemble Techniques in Social Applications,” Expert Systems with Applications, vol. 77, pp. 236-246, 2017. https://doi.org/10.1016/j.eswa.2017.02.002

[5] Atha S. and Bolla B., “Do Deep Learning Models and News Headlines Outperform Conventional Prediction Techniques on Forex Data?,” in Proceedings of Lecture Notes in Networks and Systems, Singapore, vol. 427, pp. 413-423, 2022. https://doi.org/10.1007/978-981-19-1018-0_35

[6] Behera R., Jena M., Rath S., and Misra S., “Co- LSTM: Convolutional LSTM Model for Sentiment Analysis in Social Big Data,” Information Processing and Management, vol. 58, no. 1, pp. 102435, 2021. https://doi.org/10.1016/j.ipm.2020.102435

[7] Bingler J., Kraus M., Leippold M., Webersinke N., “Cheap Talk and Cherry-Picking: What Climatebert Has to Say on Corporate Climate Risk Disclosures,” Finance Research Letters, vol. 47, pp. 1-8, 2022. https://doi.org/10.1016/j.frl.2022.102776

[8] Cammel S., De Vos MS, Van Soest D., Hettne K., and et al., “How to Automatically Turn Patient Experience Free-Text Responses into Actionable Insights: A Natural Language Programming (NLP) Approach,” BMC Medical Informatics and Decision Making, vol. 20, no. 1, 2020. https://doi.org/10.1186/s12911-020-1104-5

[9] Chia Y., Witteveen S., and Andrews M., “Transformer to CNN: Label-Scarce Distillation for Efficient Text Classification,” arXiv Preprint, vol. arXiv:1909.03508v1, pp. 1-5, 2019. https://doi.org/10.48550/arXiv.1909.03508

[10] Crowdflower., “Crowdflower:sentiment-analysis- in-text,” . https://data.world/crowdflower/sentiment- analysis-in-text/activity

[11] Da J., Forbes M., Zellers R., Zheng A., and et al., “Edited Media Understanding: Reasoning about Implications of Manipulated Images,” arXiv Preprint, vol. arXiv:2012.04726v2, pp. 1-14, 2025. https://doi.org/10.48550/arXiv.2012.04726

[12] Demszky D., Movshovitz-Attias D., Ko J., Cowen A., and et al., “GoEmotions: A Dataset of Fine- Grained Emotions,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, pp. 4040- 4054, 2020. https://doi.org/10.18653/v1/2020.acl- main.372

[13] Devlin J., Chang M., Lee K., and Toutanova K., “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minnesota, pp. 4171-4186, 2019. https://doi.org/10.18653/v1/N19-1423

[14] Du J., Ott M., Li H, Zhou X., and Stoyanov V., “General Purpose Text Embeddings from Pre- Trained Language Models for Scalable Inference,” in Proceedings of the Association for Computational Linguistics Findings: EMNLP, pp. 3018-3030, 2020. https://doi.org/10.18653/v1/2020.findings- emnlp.271

[15] Ferreira C., Robertson J., Chohan R., Pitt L., and Foster T., “The Writing is on the Wall: Predicting Customers’ Evaluation of Customer-Firm Interactions Using Computerized Text Analysis,” Journal of Service Theory and Practice, vol. 33, no. 2, pp. 309-327, 2023. https://doi.org/10.1108/JSTP-04-2022-0100

[16] Fu D., Zhu Y., Liu Z., Zheng L., and et al., “Climatebench-M: A Multi-Modal Climate Data Benchmark with A Simple Generative Method,” in Proceedings of the 34th ACM International Conference on Information and Knowledge ManagementAssociation for Computing Machinery, Birmingham, pp. 6367-6371, 2025. https://doi.org/10.48550/arXiv.2504.07394

[17] Gao Z., Feng A., Song X., and Wu X., “Target- Dependent Sentiment Classification with BERT,” IEEE Access, vol. 7, pp.154290-154299, 2019. https://doi.org/10.1109/ACCESS.2019.2946594

[18] Gaur L., Singh G., Solanki A., Jhanjhi Noor., and et al., “Disposition of Youth in Predicting Sustainable Development Goals Using the Neuro- Fuzzy and Random Forest Algorithms,” Human- Centric Computing and Information Sciences, vol. Distilled Transformer for Climate Sentiment Analysis on Social Media 439 11, pp. 2-19, 2021. https://doi.org/10.22967/HCIS.2021.11.024

[19] Hartmann J., “Emotion English DistilRoBERTa- Base,” Hugging Face, 2021. https://huggingface.co/jhartmann/emotion- english-distilroberta-base

[20] Jain P., Saravanan V., and Pamula R., “A Hybrid CNN-LSTM: A Deep Learning Approach for Consumer Sentiment Analysis Using Qualitative User-Generated Contents,” ACM Transactions on Asian and Low-Resource Language Information Processing, vol. 20, no. 5, pp. 1-15, 2021. https://doi.org/10.1145/3457206

[21] Jiang Z., Araki J., Ding H., and Neubig G., “How Can We Know When Language Models Know?,” arXiv Preprint, vol. arXiv:2012.00955v2, pp. 1- 16, 2021. https://doi.org/10.48550/arXiv.2012.00955

[22] Joachims T., “Text Categorization with Support Vector Machines: Learning with Many Relevant Features,” in Proceedings of the European conference on Machine Learning, Berlin, pp. 335- 342, 1998. https://doi.org/10.1515/9780691186740-014

[23] Joshi R., Goel P., and Joshi R., “Deep Learning �I�R�U� �+�L�Q�G�L��7�H�

[�W� �&�O�D�V�V�L�I�L�F�D�W�L�R�Q�� �$� �&�R�P�S�D�U�L�V�R�Q��´�in Proceedings of the Lecture Notes in Computer Science, Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), pp. 94-101, 2020. https://doi.org/10.1007/978-3-030-44689- 5_9

[24] Kar A., “What Affects Usage Satisfaction in Mobile Payments? Modelling User Generated Content to Develop the “Digital Service Usage �6�D�W�L�V�I�D�F�W�L�R�Q� �0�R�G�H�O��´�Information Systems Frontiers, vol. 23, no. 5, pp. 1341-1361, 2021. https://doi.org/10.1007/s10796-020-10045-0

[25] Khafajeh H., “Cyberbullying Detection in Social �1�H�W�Z�R�U�N�V� �8�V�L�Q�J� �'�H�H�S� �/�H�D�U�Q�L�Q�J��´�The International Arab Journal of Information Technology, vol. 21, no. 6, pp. 1054-1063, 2024. https://doi.org/10.34028/iajit/21/6/9

[26] Khan F., Khan S., Shamim K., Gupta Y., and Sherwani S., “Analysing Customers’ Reviews and Ratings for Online Food Deliveries: A Text Mining Approach,” International Journal of Consumer Studies, vol. 47, no. 3, pp. 953-976, 2023. https://doi.org/10.1111/ijcs.12877

[27] Kim Y., “Convolutional Neural Networks for Sentence Classification,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, Doha, pp. 1746-1751, 2014. https://doi.org/10.3115/v1/D14-1181

[28] Kumar S., Khan M., Hasanat M., Saudagar A., and et al., “Sigmoidal Particle Swarm Optimization for Twitter Sentiment Analysis,” Computers, Materials and Continua, vol. 74, no. 1, pp. 897- 914, 2023. https://doi.org/10.32604/cmc.2023.031867

[29] Lakatos E., Cioca L., Dan V., Ciomos A., and et al., “Studies and Investigation about the Attitude Towards Sustainable Production, Consumption and Waste Generation in Line with Circular Economy in Romania,” Sustainability (Switzerland), vol. 10, no. 3, pp. 1-25, 2018. https://doi.org/10.3390/su10030865

[30] Li S., Ao X., Pan F., and He Q., “Learning Policy Scheduling for Text Augmentation,” Neural Networks, vol. 145, pp. 121-127, 2022. https://doi.org/10.1016/j.neunet.2021.09.028

[31] Li W., Liu P., Zhang Q., and Liu W., “An Improved Approach for Text Sentiment Classification Based on a Deep Neural Network a Sentiment Attention Mechanism,” Future Internet, vol. 11, no. 4, pp. 1-15, 2019. https://doi.org/10.3390/FI11040096

[32] Liu Z., Liao H., Li M., Yang Q., and Meng F., “A Deep Learning-Based Sentiment Analysis Approach for Online Product Ranking with Probabilistic Linguistic Term Sets,” IEEE Transactions on Engineering Management, vol. 71, pp. 6677-6694, 2024. https://doi.org/10.1109/TEM.2023.3271597

[33] Loureiro M. and Allo M., “Sensing Climate Change and Energy Issues: Sentiment and Emotion Analysis with Social Media in the U.K. and Spain,” Energy Policy, vol. 143, pp. 111490, 2020. https://doi.org/10.1016/j.enpol.2020.111490

[34] Marlon J., Wang X., Bergquist P., Howe P., Leiserowitz A., and et al., “Change in US State- Level Public Opinion about Climate Change: 2008-2020,” Environmental Research Letters, vol. 17, no. 12, pp. 2-19, 2022, DOI:10.1088/1748-9326/aca702.

[35] Mohammad S., Bravo-Marquez F., Salameh M., and Kiritchenko S., “SemEval-2018 Task 1: Affect in Tweets,” in Proceedings of the 12th Workshop on Semantic Evaluation, New Orleans, pp. 1-17, 2018. https://doi.org/10.18653/v1/s18- 1001

[36] Mondal A., Zhu Y., Bhagat K., and Giacaman N., “Analysing User Reviews of Interactive Educational Apps: a Sentiment Analysis Approach,” Interactive Learning Environments, vol. 32, no. 1, pp. 355-372, 2024. https://doi.org/10.1080/10494820.2022.2086578

[37] Mozes M., Stenetorp P., Kleinberg B., and Griffin L., “Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples,” in Proceedings of the EACL 16th Conference of the European Chapter of the Association for Computational Linguistics, Onlin, pp. 171-186, 2021. https://doi.org/10.18653/v1/2021.eacl- main.13 440 The International Arab Journal of Information Technology, Vol. 23, No. 3, May 2026

[38] Onan A., “Bidirectional Convolutional Recurrent Neural Network Architecture with Group-Wise Enhancement Mechanism for Text Sentiment Classification,” Journal of King Saud University- Computer and Information Sciences, vol. 34, no. 5, pp. 2098-2117, 2022. https://doi.org/10.1016/j.jksuci.2022.02.025

[39] Palomares I., Martinez-Camara E., Montes R., Garcia-Moral P., Chiachio M., and et al., “A Panoramic View and Swot Analysis of Artificial Intelligence for Achieving the Sustainable Development Goals by 2030: Progress and Prospects,” Applied Intelligence, vol. 51, no. 9, pp. 6497-6527, 2021. https://doi.org/10.1007/s10489- 021-02264-y

[40] Pang B., Lee L., and Vaithyanathan S., “Thumbs up? Sentiment Classification Using Machine Learning Techniques,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 79-86, 2002. https://doi.org/10.3115/1118693.1118704

[41] Peters M, Ruder S, and Smith N., “To Tune or not to tune? Adapting Pretrained Representations to Diverse Tasks,” in Proceedings of the 4th Workshop on Representation Learning for NLP , Florence, pp. 7-14, 2019. https://doi.org/10.18653/v1/w19-4302

[42] Pirouz B., Haghshenas S., Pirouz B., Haghshenas S., and Piro P., “Development of an Assessment Method for Investigating the Impact of Climate and Urban Parameters in Confirmed Cases of COVID-19: a New Challenge in Sustainable Development,” International Journal of Environmental Research and Public Health, vol. 17, no. 8, 2020. https://doi.org/10.3390/ijerph17082801

[43] Poria S., Hazarika D., Majumder N., Naik G., and et al., “MELD: a Multimodal Multi-Party Dataset for Emotion Recognition in Conversations,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp.527-536, 2019. https://doi.org/10.18653/v1/p19-1050

[44] Radford A., Narasimhan K., Salimans T., and Sutskever I., Improving Language Understanding by Generative Pre-Training, https://cdn.openai.com/research-covers/language- unsupervised/language_understanding_paper.pdf, Last Visited, 2025.

[45] Rinzin C., Vermeulen W., and Glasbergen P., “Public Perceptions of Bhutan’s Approach to Sustainable Development in Practice,” Sustainable Development, vol. 15, no. 1, pp. 52- 68, 2007. https://doi.org/10.1002/sd.293

[46] Sanh V., Debut L., Chaumond J., and Wolf T., “Distilbert, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter,” arXiv Preprint, vol. arXiv:1910.01108v4, pp. 1-5, 2019.

[47] Saravia E., Liu H., Huang Y., Wu J., and Chen Y., “Carer: Contextualized Affect Representations for Emotion Recognition,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, Brussels, pp. 3687-3697, 2018. https://doi.org/10.18653/v1/d18-1404

[48] Scherer K. and Wallbott H., “Evidence for Universality and Cultural Variation of Differential Emotion Response Patterning.,” Journal of Personality and Social Psychology, vol. 66, no. 2, pp. 310-328, 1994. https://psycnet.apa.org/doi/10.1037/0022- 3514.66.2.310

[49] Schmidt S., Zorenböhmer C., Arifi D., and Resch B., “Polarity-Based Sentiment Analysis of Georeferenced Tweets Related to the 2022 Twitter Acquisition,” Information, vol. 14, no. 2, pp. 1-11, 2023. https://doi.org/10.3390/info14020071

[50] Shrivastava K., Kumar S., and Jain D., “An Effective Approach for Emotion Detection in Multimedia Text Data Using Sequence Based Convolutional Neural Network,” Multimedia Tools and Applications, vol. 78, no. 20, pp. 29607- 29639, 2019. https://doi.org/10.1007/s11042-019- 07813-9

[51] Singh J., Kumar A., Rana N., and Dwivedi Y., “Attention-Based LSTM Network for Rumor Veracity Estimation of Tweets,” Information Systems Frontiers, vol. 24, pp. 459-474, 2022. https://doi.org/10.1007/s10796-020-10040-5

[52] Singla S. and Ramachandra N., “Comparative Analysis of Transformer Based Pre-Trained NLP Models,” International Journal of Computer Sciences and Engineering, vol. 8, no. 11, pp. 40- 44, 2020. https://doi.org/10.26438/ijcse/v8i11.4044

[53] Socher R., Perelygin A., Wu J., Chuanga J., and et al., “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA no. October, pp. 1631-1642, 2013. https://aclanthology.org/D13-1170/

[54] Thiengburanathum P. and Charoenkwan P., “SETAR: Stacking Ensemble Learning for Thai Sentiment Analysis Using RoBERTa and Hybrid Feature Representation,” IEEE Access, vol. 11, pp. 92822-92837, 2023. https://doi.org/10.1109/ACCESS.2023.3308951

[55] Ullah A., Khan S., and Nawi N., “Review on Sentiment Analysis for Text Classification Techniques From 2010 to 2021,” Multimedia Tools and Applications, vol. 82, no. 6, pp. 8137- 8193, 2023. https://doi.org/10.1007/s11042-022- 14112-3

[56] Upadhyaya A., Fisichella M., and Nejdl W., “A Multi-task Model for Emotion and Offensive Aided Stance Detection of Climate Change Distilled Transformer for Climate Sentiment Analysis on Social Media 441 Tweets,” in Proceedings of the ACM Web Conference, Austin, pp. 3948-3958, 2023. https://doi.org/10.1145/3543507.3583860

[57] Vashishtha S., Gupta V., and Mittal M., “Sentiment Analysis Using Fuzzy Logic: A Comprehensive Literature Review,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 13, no. 5, pp. 1-37, 2023. https://doi.org/10.1002/widm.1509

[58] Webersinke N., Kraus M., Bingler J., and Leippold M., “ClimateBert: A Pretrained Language Model for Climate-Related Text,” arXiv Preprnit, vol. arXiv:2110.12010v3, pp. 1-9, 2022. https://doi.org/10.48550/arXiv.2110.12010

[59] Wortsman M., Ilharco G., Gadre S., Roelofs R., and et al., “Model Soups: Averaging Weights of Multiple Fine-Tuned Models Improves Accuracy Without Increasing Inference Time,” in Proceedings of the Machine Learning Research, Baltimore, vol. 162, pp. 23965-23998, 2022. https://doi.org/10.48550/arXiv.2203.05482

[60] Xiao Y., Li C., Thurer M., Liu Y., and Qu T., “Towards Lean Automation: Fine-Grained Sentiment Analysis for Customer Value Identification,” Computers and Industrial Engineering, vol. 169, pp. 1-10, 2022. https://doi.org/10.1016/j.cie.2022.108186

[61] Younisse R., Awajan A., and Younes M., “A New Data Reduction Technique for Efficient Arabic Data Sentiment Analysis,” The International Arab Journal of Information Technology, vol. 22, no. 5, pp. 930-939, 2025. https://doi.org/10.34028/iajit/22/5/7