Cross-domain sentiment classification transfers the knowledge from the source domain to the target domain lacking supervised information for sentiment classification. Existing cross-domain sentiment classification methods establish connections by extracting domain-invariant features manually. However, these methods have poor adaptability to bridge connections across different domains and ignore important sentiment information. Hence, we propose a Topic Lite Bidirectional Encoder Representations from Transformers (T-LBERT) model with domain adaption to improve the adaptability of cross-domain sentiment classification. It combines the learning content of the source domain and the topic information of the target domain to improve the domain adaptability of the model. Due to the unbalanced distribution of information in the combined data, we apply a two-layer attention adaptive mechanism for classification. A shallow attention layer is applied to weigh the important features of the combined data. Inspired by active learning, we propose a deep domain adaption layer, which actively adjusts model parameters to balance the difference and representativeness between domains. Experimental results on Amazon review datasets demonstrate that the T-LBERT model considerably outperforms other state- of-the-art methods. T-LBERT shows stable classification performance on multiple metrics.

[1] Beigi O. and Moattar M., “Automatic Construction of Domain-Specific Sentiment Lexicon for Unsupervised Domain Adaptation and Sentiment Classification,” Knowledge-Based Systems, vol. 213, pp. 106423-106426, 2021.

[2] Blei D., Ng Y., and Jordan M., “Latent Dirichlet Allocation,” Journal of Machine Learning Research, vol. 3, pp. 993-1022, 2003.

[3] Blitzer J., Dredze M., and Pereira F., “Biographies, Bollywood, Boom-Boxes and Blenders: Domain Adaptation for Sentiment Classification,” in Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, pp. 440-447, 2007.

[4] Cambria E., Das D., Bandyopadhyay S., and Feraco A., Affective Computing and Sentiment Analysis, A Practical Guide to Sentiment Analysis, Springer, 2017.

[5] Chen T., Liao Y., Chuang C., Hsu W., Fu J., and Sun M., “Show, Adapt and Tell: Adversarial Training of Cross-Domain Image Captioner,” in Proceedings of the IEEE International Conference on Computer Vision, Venice, pp. 521-530, 2017.

[6] Devlin J., Chang M., Lee K., and Toutanova K., “Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding,” arXiv preprint arXiv:1810.04805, 2018.

[7] Du Y., He M., Wang L., and Zhang H., “Wasserstein Based Transfer Network for Cross- Domain Sentiment Classification,” Knowledge- Based Systems, vol. 204, pp. 106162-106178, 2020.

[8] Ganin Y., Ustinova E., Ajakan H., Germain P., Larochelle H., Laviolette F., Marchand M., and Lempitsky V., “Domain-Adversarial Training of Neural Networks,” The Journal of Machine Learning Research, vol. 17, no.1, pp. 2096-2030, 2016.

[9] Jia X., Jin Y., Li N., Su X., Cardiff B., and Bhanu B., “Words Alignment Based on Association Rules for Cross-Domain Sentiment Classification,” Frontiers of Information Technology and Electronic Engineering, vol. 19, no. 2, pp. 260-272, 2018.

[10] Jiang J. and Zhai C., “Instance Weighting for Domain Adaptation in NLP,” in Proceedings of the 45th Annual Meeting of The Association of Computational Linguistics, Prague, pp. 264-271, T-LBERT with Domain Adaptation for Cross-Domain Sentiment Classification 149 2007.

[11] Khondaker A., Khandaker A., and Uddin J., “Computer Vision-based Early Fire Detection Using Enhanced Chromatic Segmentation and Optical Flow Analysis Technique,” The International Arab Journal of Information Technology, vol. 17, no. 6, pp. 947-953, 2020.

[12] Lan Z., Chen M., Goodman S., Gimpel K., Sharma P. and Soricut R., “Albert: A lite bert for Self-Supervised Learning of Language Representations,” arXiv preprint arXiv:1909.11942, 2019.

[13] Li Y., Xiao F., Chen Z., and Li B., “Adaptive Active Learning for Semi-Supervised Learning,” Ruan Jian Xue Bao/Journal of Software, vol. 31, no. 12, pp. 3808-3822, 2020.

[14] Liu B., “Sentiment Analysis and Opinion Mining,” Synthesis Lectures on Human Language Technologies, vol. 5, no. 1, pp. 1-167, 2012.

[15] Meng J., Dong Y., Long Y., and Zhao D., “An Attention Network Based on Feature Sequences for Cross-Domain Sentiment Classification,” Intelligent Data Analysis, vol. 25, no. 3, pp. 627- 640, 2021.

[16] Myagmar B., Li J., and Kimura S., “Cross- Domain Sentiment Classification with Bidirectional Contextualized Transformer Language Models,” IEEE Access, vol. 7, pp. 163219-163230, 2019.

[17] Ostendorff M., Bourgonje P., Berger M., Moreno-Schneider J., Rehm G., and Gipp B., “Enriching Bert With Knowledge Graph Embeddings for Document Classification,” arXiv preprint arXiv:1909.08402, 2019.

[18] Pan S., Ni X., Sun J., Yang Q., and Chen Z., “Cross-Domain Sentiment Classification Via Spectral Feature Alignment,” in Proceedings of the 19th International Conference on World Wide Web, Raleigh, pp. 751-760, 2010.

[19] Qu X., Zou Z., Cheng Y., Yang Y., and Zhou P., “Adversarial Category Alignment Network for Cross-Domain Sentiment Classification,” in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, pp. 2496-2508, 2019.

[20] Ramshankar N. and Prathap P., “A Novel Recommendation System Enabled by Adaptive Fuzzy Aided Sentiment Classification for E- Commerce Sector Using Black Hole-Based Grey Wolf Optimization,” Sādhanā, vol. 46, no. 3, pp. 1-24, 2021.

[21] Sener O., Song H., Saxena A., and Savarese S., “Learning Transferrable Representations for Unsupervised Domain Adaptation,” Advances in Neural Information Processing Systems, vol. 29, pp. 2110-2118, 2016.

[22] Settles B., “Active Learning Literature Survey,” Technical Report, University of Wisconsin, 2009.

[23] Sharma R., Bhattacharyya P., Dandapat S., and Bhatt H., “Identifying Transferable Information Across Domains for Cross-Domain Sentiment Classification,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, pp. 968-978, 2018.

[24] Tang D., Qin B., and Liu T., “Document Modeling with Gated Recurrent Neural Network for Sentiment Classification,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing, Lisbon, pp. 1422-1432, 2015.

[25] Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A., Kaiser Ł., and Polosukhin I., “Attention is All You Need,” in Advances in Neural Information Processing Systems, Long Beach, pp. 5998-6008, 2017.

[26] Wang L., Niu J., Song H., and Atiquzzaman M., “Sentirelated: A Cross-Domain Sentiment Classification Algorithm for Short Texts through Sentiment Related Index,” Journal of Network and Computer Applications, vol. 101, pp. 111- 119, 2018.

[27] Wei X., Lin H., Yu Y., and Yang L., “Low- Resource Cross-Domain Product Review Sentiment Classification Based on A CNN with an Auxiliary Large-Scale Corpus,” Algorithms, vol. 10, no.3, pp. 81, 2017.

[28] Wu F., Huang Y., and Yan J., “Active Sentiment Domain Adaptation,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, pp. 1701- 1711, 2017.

[29] Yan X., Guo J., Lan Y., and Cheng X., “A Biterm Topic Model for Short Texts,” in Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, pp. 1445- 1456, 2013.

[30] Yu J., Elkaref M., and Bohnet B., “Domain Adaptation for Dependency Parsing Via Self- Training,” in Proceedings of the 14th International Conference on Parsing Technologies, Bilbao, pp. 1-10, 2015.

[31] Zadrozny B., “Learning and Evaluating Classifiers Under Sample Selection Bias,” in Proceedings of the 20st International Conference On Machine Learning, Banff Alberta, pp. 114, 2004.

[32] Zhao C., Wang S., and Li D., “Multi-Source Domain Adaptation with Joint Learning for Cross-Domain Sentiment Classification,” Knowledge-Based Systems, vol. 191, pp. 105254- 1053.13, 2020.

[33] Zhao C., Wang S., Li D., Liu X., Yang X., and Liu J., “Cross-domain Sentiment Classification Via Parameter Transferring and Attention 150 The International Arab Journal of Information Technology, Vol. 20, No. 1, January 2023 Sharing Mechanism,” Information Sciences, vol. 578, pp. 281-296, 2021.